Logit and Sigmoid Functions and its Calculus

Logit and Sigmoid Functions and its Calculus

Definitions

The sigmoid function is defined as:

sigmoid(x) = 1 ÷ (1 + e^(-x))

The logit function is defined as:  

logit(p) = ln (p ÷ (1 - p))

For logit(p) to have a real number answer, 0 ≤ p < 1

Transform from the Sigmoid Function to the Logit Function

We can easily transform from the sigmoid function to the logit function.  

Let s = sigmoid(x). Then:

s = 1 ÷ (1 + e^(-x))

s * (1 + e^(-x)) = 1

s + s * e^(-x) = 1

s * e^(-x) = 1 - s

e^(-x) = (1 - s) ÷ s

e^x = s ÷ (1 - s)

x = ln(s ÷ (1 - s)) = logit(s)

To transform from the logit function to the sigmoid function, just go backwards.  

Sigmoid Function:  Derivative and Integral

Derivative

s = sigmoid(x)

s = 1 ÷ (1 + e^(-x))

Using the quotient rule of derivatives:

ds/dx = [(1 + e^(-x)) * 0 - 1 * -e^(-x)] ÷ (1 + e^(-x))^2

= -(-e^(-x)) ÷ (1 + e^(-x))^2

= -e^(-x) ÷ (1 + e^(-x))^2

Integral

s = sigmoid(x)

s = 1 ÷ (1 + e^(-x))

Multiply both sides by e^x ÷ e^x:

s * (e^x ÷ e^x) = (e^x ÷ e^x) * (1 ÷ (1 + e^(-x)))

s = e^x ÷ (e^x + 1)

Integral:

∫ e^x ÷ (e^x + 1) dx

Let u = e^x + 1.  Then du = e^x dx 

= ∫  du ÷ (u + 1) 

= ln (u + 1) + C

= ln (e^x + 1) + C

Summary:

d/dx sigmoid(x) = -e^(-x) ÷ (1 + e^(-x))^2

∫ sigmoid(x) dx = ln (e^x + 1) + C

Logit Function:  Derivative and Integral

Derivative

logit(p) = ln (p ÷ (1 - p))

L = ln (p ÷ (1 - p))

Derivative:

dL/dp =  [(1 - p) ÷ p] * d/dp ln (p ÷ (1 - p))

=  [(1 - p) ÷ p] * [(1 - p) * 1 - p * (-1)] ÷ [(1 - p)^2] 

=  [(1 - p) ÷ p] * [1 - p + p] ÷ [(1 - p)^2]

=  [(1 - p) ÷ p] * 1 ÷ (1 - p)^2

= 1 ÷ [p * (1 - p)]

Integral:

∫ ln (p ÷ (1 - p)) dp

By integration by parts:

u = ln (p ÷ (1 - p)) 

du = 1 ÷ [p * (1 - p)] dp

v = dp

v = p

Then:

∫u dv

= p * ln ( p ÷ (1 - p)) - ∫ p ÷ (1 - p) dp

= p * ln ( p ÷ (1 - p)) + ∫ -p ÷ (1 - p) dp

= p * ln ( p ÷ (1 - p)) + ln(1 - p) + C

In Summary:

d/dp logit(p) = 1 ÷ [p * (1 - p)]

∫ logit(p) dp = p * ln ( p ÷ (1 - p)) + ln(1 - p) + C

Eddie

All original content copyright, © 2011-2022.  Edward Shore.   Unauthorized use and/or unauthorized distribution for commercial purposes without express and written permission from the author is strictly prohibited.  This blog entry may be distributed for noncommercial purposes, provided that full credit is given to the author. 

Python - Lambda Week: Integration by Simpson's Rule

Python - Lambda Week: Integration by Simpson's Rule

Welcome to Python Week!  This we we're going to cover calculus and the keyword lambda.

Note:  All Python scripts presented this week were created using a TI-NSpire CX II CAS.   As of June 2022, the lambda keyword is available on all calculators (in the United States) that have Python.   If you are not sure, please check your calculator manual. 

Simpson's Rule

The Simpson's Rule estimates numeric integrals by:

∫( f(x) dx, x = a to b) ≈

(b - a) /(3 * n) * (f(a) + 4 * f1 + 2 * f2 + 4 * f3 + .... + 2 * f_n-2 + 4 * f_n-1 + f(b))

n must be an even number of partitions.  The more partitions, the higher the accuracy and the higher computation time.

integrallam.py:  Numeric Integer

from math import *

print("The math module is imported.")

print("Integra of f(x), 6 places")

f=eval("lambda x:"+input("f(x)? "))

# input parameters

a=eval(input("lower = "))

b=eval(input("upper = "))

n=int(input("even parts: "))

# checksafe, add 1 if n is odd

if n/2-int(n/2)==0:

  n=n+1

# integral calculus

s=f(a)+f(b)

w=1

# 1 to n-1

for i in range(1,n):

  w=f(a+i*(b-a)/n)

  s+=(2*w) if (i/2-int(i/2)==0) else (4*w)

s*=(b-a)/(3*n)

print("Integral: "+str(round(s,6)))

All original content copyright, © 2011-2022.  Edward Shore.   Unauthorized use and/or unauthorized distribution for commercial purposes without express and written permission from the author is strictly prohibited.  This blog entry may be distributed for noncommercial purposes, provided that full credit is given to the author.