The basics of Logarithms – with examples

Tue, 06 Feb 2018 00:00:00 +0000

Logarithms are widely used in computer science (e.g. for algorithm analyses, floating point number limitations, scaling data, feature transformations). Not coming from a mathematics background (I don’t!) logarithms can seem confusing at first.

Let’s look at the very basics of logarithms to get an understanding of how they can be broken apart, some of the properties that can be utilized, and why they work.

What is a Logarithm?

$$\log_{b}(x)$$

The logarithm is defined as the inverse operation to exponentiation. To get the logarithm of a number, we need to find out, to what exponent another number, called the base, needs to be raised to produce that first number.

Let’s look at an example:

$$\log_{10} 100$$

To rephrase the last sentence again: To get the logarithm of a number 100, we need to find out, to what exponent another number, 10, called the base, needs to be raised to produce that first number 100.

Since $10^2 = 100$, the logarithm of 100, base 10, is 2:

$$\log_{10} 100 = 2$$

In Python we would write this as:

>>> math.log10(100)
2.0
>>> 10**2 # "reverse" the operation
100

And to bring it into a general form:

$$\log_{b} x = y$$$$b^y = x$$

Note that $\log_{b} x$ only returns real numbers for x >0 (as we will also see on the graph further down).

Now let’s come back to the first sentence of this section: The logarithm is the inverse operation to exponentiation. In other words, it undoes the exponentiation operation.

We can write some code to see this with sample numbers (math.exp(x) returns e to the power of x, math.log(x) returns the logarithm of x in base e).

>>> x = 2.0
>>> math.log(x), math.log(math.exp(x))
(0.6931471805599453, 2.0)
# ==> log(exp(x)) = x

When we plot the two functions, we can see that $\log (x)$ (i.e. $\log_{e}x$) is reflecting $exp(x)$ (i.e. $e^x$).

The base

You will most often see logarithms with base e, base 10 or base 2, but the base can be any positive number not equal to 1.

In Python, the most common log bases are available as a function of the math module:

>>> math.log10(100) # base 10
2.0
>>> math.log(100) # base e, or natural log, or ln
4.605170185988092
>>> math.log2(100) # base 2
6.643856189774724
>>> math.log(100, 5) # base 5, or any desired base as second argument
2.8613531161467867

One important property is, that the logarithm of 1 is zero for any base. We can understand this by taking a log apart like in the first section:

$$\log_{10} 1 $$

10 raised to what power is 1? Only $10^0$ will equal 1, and the same is true for any other base number. If we plot the ln (log e), log10 and log2, it becomes more clear: for all bases, y is 0 where x is 1.

import numpy as np
import matplotlib.pyplot as plt

x = np.linspace(0.001, 16, 2000)
y_e = np.log(x)
y_10 = np.log10(x)
y_2 = np.log2(x)

plt.plot(x, y_e)
plt.plot(x, y_10)
plt.plot(x, y_2)

plt.legend(['ln', 'log10', 'log2'], loc='lower right')
plt.xticks(range(math.floor(min(x)), math.ceil(max(x))+1))
plt.axhline(0, color='black', linewidth='0.5')
plt.axvline(0, color='black', linewidth='0.5')

Another visible attribute is that when base and argument are the same number, the result is always 1. Here is the same graph zoomed in a little bit and with grid lines. You can see that for log2 at x = 2, y equals 1, and for the natural log with base e (~ 2.7) and x at e, y also equals 1.

Going to the left side on the x axis, log(x) becomes negative for x < 1 for any base, and diverges towards negative infinity as x approaches zero. The formal way of writing this is:

$$\lim_{x \to 0} \log_{b} x = - \infty$$

In the other direction, as x approaches infinity, log(x) diverges towards positive infinity for any base:

$$\lim_{x \to \infty} \log_{b} x = \infty$$

Note: When log is written without a base, it usually refers to $\log_{2}(x)$ in computer science algorithms. In programming/mathematics/machine learning it generally refers to the natural logarithm (e.g. Python math.log or numpy.log), i.e. base e. Base 10 is also commonly implied when you just have $\log (x)$. 🤔 This is kind of confusing and often just depends on the context. See Wikipedia for more info on particular bases.

Logarithmic rules

Logarithms have important properties, or laws, that one should be familiar with and make computations easier. Let’s have a look at three of them and also understand why they work:

Product rule

$$\log_{b}(x \cdot y) = \log_{b}(x) + \log_{b}(y)$$

The logarithm of a product (multiplication of x and y), can be broken apart and rewritten as the sum of the logarithm of x and the logarithm of y (addition). To see why this formula works, we need to know a property of exponents. Let’s take a look at an example:

$$\log_{2}(8 \cdot 16) = \log_{2}(8) + \log_{2}(16)$$

We can substitute 8 and 16 as powers of the base, 2.

$$\log_{2}(2^3 \cdot 2^4)$$

Using a property of exponents, that says $a^m \cdot a^n = a^{m+n}$, we can rewrite this as:

$$\log_{2}(2^{3+4})$$

Now we need to find the power to which we need to raise 2, the base, to get $2^{3+4}$. This is obviously 3 + 4 itself. So let’s write this down:

$$3+4$$

Turning this back into logarithmic form, we get:

$$\log_{2}(8 \cdot 16) = \log_{2}(8) + \log_{2}(16)$$

Power rule 🚀

$$\log_{b}(x^y) = y \cdot \log_{b}(x)$$

With that name it better be good 🤓. And it is. When we have the log of a number raised to a power (here: $x^y$), we can bring the exponent down and in front of the log. So for the above equation, it means that the logarithm of x raised to the power of y is equal to y times the logarithm of x. Let’s see why this works by looking at an example:

$$\log_{10}(2^6) = 6 \cdot \log_{10}(2)$$

Another way of looking at this problem is:

$$\log_{10}(2 \cdot 2 \cdot 2 \cdot 2 \cdot 2 \cdot 2)$$

From above (product rule) we know that we can turn multiplication into addition. So let’s do that:

$$\log_{10}(2) + \log_{10}(2) + \log_{10}(2) + \log_{10}(2) + \log_{10}(2) + \log_{10}(2)$$

And this in turn can be simplified to:

$$\log_{10}(2^6) = 6 \cdot \log_{10}(2)$$

Quotient Rule

$$\log_{b}(\frac{x}{y}) = \log_{b}(x) - \log_{b}(y)$$

Just like with the multiplication to addition (product rule), we can turn devision into subtraction.

We will make up an example and see why this works:

$$\log_{10}(\frac{5}{10}) = \log_{10}(5) - \log_{10}(10)$$

We know that $\frac{5}{10}$ is the same as $5 \cdot \frac{1}{10}$ is the same as $5 \cdot 10^{-1}$. So let’s write out the log again:

$$\log_{10}(\frac{5}{10}) = \log_{10}(5 \cdot 10^{-1})$$

Using the product rule (see above), we can turn this into:

$$\log_{10}(5) + \log_{10}(10^{-1})$$

And now, using the power rule, we move the -1 down and in front:

$$\log_{10}(5) + -1 \cdot \log_{10}(10)$$

So we get:

$$\log_{10}(\frac{5}{10}) = \log_{10}(5) - \log_{10}(10)$$

Change of base

In Python we can calculate the log in any base with math.log(x, base). When you don’t have that function available (like on a calculator), you might want to know how you can easily change the base to a supported one.

The general rule is:

$$\log_{b}(x) = \frac{\log_{c}(x)}{\log_{c}(b)}$$

But let’s look at an example again (from base 2 to base 10) and see why this works.

$$\log_{2}(16) = x$$

For this equation we know that $2^x = 16$.

We will now take the log in our desired base 10 on both sides, to be able to use the power rule.

$$\log_{10}(2^x) = \log_{10}(16)$$

With the power rule we can bring the exponent down and to the front:

$$x \cdot \log_{10}(2) = \log_{10}(16)$$

To get out final result, we devide both sides by $\log_{10}(2)$:

$$x = \frac{\log_{10}(16)}{\log_{10}(2)} = \log_{2}(16)$$

Much more to cover, but that’s it for now at this point. Take a look at the handy cheat sheet of logarithmic identities for a review of the above, plus additional properties.

LaTeX mathematics cheat sheet

Mon, 12 Jun 2017 00:00:00 +0000

LaTeX is the de facto standard typesetting system for scientific writing. A lot of the nice looking equations you see in books and all around the web are written using LaTeX commands. Knowing a few of the mathematics commands is not only helpful if you want to write a book or an article (or do some extreme stuff), but can come in handy in a lot of places, as many systems support LaTeX. You can use LaTeX in MathJax to display expressions on the web (like here), you can make yourself good looking mathematics flashcards in Anki, you can even nerd out and send formulas built with LaTeX commands to your friends via an iMessage app. Also, Apple’s latest Pages release now supports LaTeX equations.

Reasons enough to get familiar with the standard commands!

While a lot of commands can be written out in plain (e.g. 1+1=2), there are other frequently used commands you will need to look up or memorise.

I plan to update this post continuously as I find myself looking for a common symbol I haven’t listed yet. Feel free to drop me an email or comment when you land here and don’t find the answer to a frequently used symbol.

Here is the cheat sheet (naturally incomplete):

Fractions

Command	Description	Output
\frac	Build a fraction like so: \frac{1}{2}	$$\frac{1}{2}$$
\frac{\frac{}}{}	You can nest fractions: \frac{\frac{1}{2}}{2}	$$\frac{\frac{1}{2}}{2}$$

Greek letters

(capitalize by capitalizing the command)

Command	Description	Output
\alpha	alpha	$$\alpha$$
\beta	beta	$$\beta$$
\gamma	gamma	$$\gamma$$
\delta	delta	$$\delta$$
\epsilon	epsilon	$$\epsilon$$
\zeta	zeta	$$\zeta$$
\eta	eta	$$\eta$$
\theta	theta	$$\theta$$
\iota	iota	$$\iota$$
\kappa	kappa	$$\kappa$$
\lambda	lambda	$$\lambda$$
\mu	mu	$$\mu$$
\nu	nu	$$\nu$$
\xi	xi	$$\xi$$
o	omicron	$$o$$
\pi	pi	$$\pi$$
\rho	rho	$$\rho$$
\sigma	sigma	$$\sigma$$
\tau	tau	$$\tau$$
\upsilon	upsilon	$$\upsilon$$
\phi	phi	$$\phi$$
\chi	chi	$$\chi$$
\psi	psi	$$\psi$$
\omega	omega	$$\omega$$

Logic

Command	Description	Output
\forall	For all	$$\forall$$
\exists	Exists	$$\exists$$
\lor	Or	$$\lor$$
\land	And	$$\land$$
\veebar	Xor	$$\veebar$$
\neg	Not	$$\neg$$

Operators

Command	Description	Output
\times	Times	$$\times$$
\cdot	Dot	$$\cdot$$
\div	Division	$$\div$$
\pm	Plus minus	$$\pm$$

Relation

Command	Description	Output
\neq	Not equal	$$\neq$$
\approx	Approximately equal	$$\approx$$
\leq	Less than or equal	$$\leq$$
\geq	Greater than or equal	$$\geq$$
\ll	Much less than	$$\ll$$
\gg	Much greater than	$$\gg$$

Sets

(Often you can put an “n” before the command and get the negation)

Command	Description	Output
\supset	Proper superset	$$\supset$$
\supseteq	Superset	$$\supseteq$$
\subset	Proper Subset	$$\subset$$
\subseteq	Subset	$$\subseteq$$
\in	Member	$$\in$$
\emptyset	Empty set	$$\emptyset$$
\mathbb{R}	Set of real numbers	$$\mathbb{R}$$
\cup	Set union (belonging to A OR B)	$$\cup$$
\cap	Set intersection (belonging to A AND B)	$$\cap$$

Super-/Subscript (Exponents / Indices)

Command	Description	Output
^	Use ^ for superscript. Example: x^2	$$x^2$$
^{}	Use ^{} for exponents with >1 digit. Example: x^{10}	$$x^{10}$$
_	Use _ for subscript. Example: x_0	$$x_0$$
_{}	Use _{} for subscript with >1 digit. Example: x_{10}	$$x_{10}$$

Others

Command	Description	Output
\infty	Infinity	$$\infty$$
\partial	Partial	$$\partial$$
\hat{}	Estimator	$$\hat{\theta}$$
\sqrt[root]{}	Square root	$$\sqrt[3]{4}$$

Mathematics on David Hamann