whetting your appetite: smart indexing¶
In [1]:
%matplotlib inline
import pandas as pd
measurement = pd.Series.from_csv('measurement.csv', header=0)
measurement.plot()
Out[1]:
In [2]:
reference = pd.Series.from_csv('reference.csv')
reference.plot()
measurement.plot()
Out[2]:
In [3]:
norm_measurement = measurement / reference
norm_measurement.plot()
Out[3]:
whetting your appetite: Gaußian uncertainty propagation & units¶
In [6]:
from uncertainties import ufloat
import pint
u = pint.UnitRegistry()
l = ufloat(2, 0.2, tag='length') * u.m
t = ufloat(7.8, 0.002, tag='time') * u.s
v = l/t
v.to(u.km/u.hour)
v.magnitude.error_components()
Out[6]:
In [7]:
l - l
Out[7]:
In [8]:
l + 2 *u.inch
Out[8]:
In [9]:
l - 2 * u.s
what is python / why this talk?¶
- scripting language: rapid prototyping
- white space is important
- core python language + ecosystem
Basics: importing modules¶
In [ ]:
import module / from module import thing
In [10]:
# example
import numpy as np
print(np .pi)
from numpy import pi
print(pi)
Basics: data types¶
In [11]:
# strings
s = "hi\nthere"
print(s)
In [12]:
# double or single quotes
s = 'hi\nthere'
print(s)
In [13]:
# triple quotes for easy multilines
s = """hi
there"""
print(s)
In [14]:
# integers, floats, complex numbers
i = 2
f = 3.6
c = 3.6 + 5.2j
i, f, c
Out[14]:
In [16]:
# lists
l = ['a', 2.3, 5]
l[1]
Out[16]:
In [17]:
# dictionaries (maps)
d = {"a": 2.5,
"b": 3.0,
"c": 15}
d["d"]
In [18]:
d["d"] = 2
d["d"]
Out[18]:
In [30]:
# pandas DataFrame
import pandas as pd
df = pd.read_csv('./example.dat', header=201, sep='\t', index_col=0)
df
Out[30]:
In [20]:
df.loc[0.5:1.2]
Out[20]:
In [31]:
df.iloc[1:3]
Out[31]:
In [32]:
df.plot(y="Keysight1", logy=True)
Out[32]:
In [38]:
# pandas Series
ser = df.iloc[:-3]
ser
Out[38]:
In [39]:
pd.Series([1.2, 2.2, 3.6], index=[3300, 3301, 3302])
Out[39]:
understanding "whetting your appetite: smart indexing"¶
In [ ]:
import pandas as pd
measurement = pd.Series.from_csv('measurement.csv', header=0)
reference = pd.Series.from_csv('reference.csv')
norm_measurement = measurement / reference
measurement.plot()
norm_measurement.plot()
digression: "dumb" arrays¶
In [42]:
l = [2, 4, 6, 8]
l/2
In [ ]:
import numpy as np
a = np.array(l)
a/2
In [46]:
3%2
Out[46]:
listvs.np.array: better math, same dtypenp.arrayvspd.Series/pd.DataFrame: smart indexing, plotting, other convenience
=> if in doubt, use pd.Series/pd.DataFrame
More core python: functions¶
In [47]:
def sq_half(x):
return x**2 / 2
sq_half(4)
Out[47]:
In [58]:
def rms(x, y=2):
"""root mean square of x and y"""
return np.sqrt((x**2 + y**2) / 2), x, y
a, b, c = rms(3)
More core python: classes + objects¶
In [59]:
import numpy as np
class Point:
"""A 2d point"""
def __init__(self, x, y):
self.x = x
self.y = y
def distance(self, other):
"""Euclidean distance to the other point."""
return np.sqrt( (self.x - other.x)**2 + (self.y - other.y)**2 )
p = Point(2, 3)
q = Point(3, 7)
p.distance(q)
Out[59]:
- use classes to group related data and functionality
understanding "whetting your appetite: uncertainties + units"¶
In [60]:
from uncertainties import ufloat
import pint
u = pint.UnitRegistry()
l = ufloat(2, 0.2, tag='length') * u.m
t = ufloat(7.8, 0.002, tag='time') * u.s
v = l/t
type(v)
Out[60]:
In [61]:
type(v.magnitude)
Out[61]:
In [63]:
v.magnitude.error_components()
Out[63]:
- ufloat objects grouping magnitude and uncertainty, quantity objects grouping magnitude and unit
- composing this leads to easy-to-use calculator
Last important bit of syntax: loops and conditionals¶
In [ ]:
for x in iterable:
do_stuff
In [65]:
l = ["a", "b", 15]
for x in l:
print(x)
In [ ]:
# to count (like loop in C/C++)
for i in range(5):
print(i)
In [69]:
range(100000)
Out[69]:
In [ ]:
# conditionals
In [ ]:
if thing:
do_stuff
In [75]:
a = 2
if a < 0:
a = a * -1
elif a < 2:
print('haha')
print(a)
In [73]:
# equivalent
a = abs(a)
Getting help¶
In [ ]:
thing?
In [79]:
rms??
- google: python + thing
- prefer readthedocs.org and docs.python.org
- show pint docs
Environment¶
ipython: interactive terminaljupyter: interactive notebooks in the browser (and slide shows)pycharm: IDE with debugger, syntax checks etc.conda: binary package manager for python: stable, easy installation
Tour of the ecosystem¶
core stdlib¶
- https://docs.python.org/3/library/index.html
- lots of facilities for standard programming, not so much data analysis
most useful usually:
os.pathloggingsubprocess
worth a look if you search something
In [ ]:
import subprocess
subprocess.run(['ls', '-la'])
In [81]:
!ls -la
numpy¶
- http://www.numpy.org/
- math
- "dumb" arrays
- matrices
In [ ]:
import numpy as np
scipy¶
- https://docs.scipy.org/doc/numpy/user/index.html
- linear algebra
- optimization (fitting)
- lots of advanced math (ex.:
sp.special.erf)
In [ ]:
import scipy as sp
pandas¶
- https://pandas.pydata.org/
- Series / DataFrame
- i/o
- advanced querying
- easy plotting
In [ ]:
import pandas as pd
PTB-interesting¶
- uncertainties: https://pythonhosted.org/uncertainties/ Uncertainty propagation
- pint: https://pint.readthedocs.io Units, unit conversion
- periodictable: http://periodictable.readthedocs.io Includes Henke tables
plotting¶
- matplotlib: https://matplotlib.org/ fine-grained control over plots (interactive + static publication quality)
- plotly: https://plot.ly/python/getting-started/#initialization-for-offline-plotting interactive publications quality plots
faster computation¶
- ipyparallel: https://ipyparallel.readthedocs.io/en/latest/ easy parallelization on single computer, "easy" distribution over all your colleague's computers
numba: http://numba.pydata.org/ make specific, math-heavy functions go fast
caveat: conversion needs some work, not all functions from numpy are available in numba
more¶
- emcee: http://dfm.io/emcee/current/ markov-chain monte carlo, e.g. sampling of random distributions, estimation of uncertainties of inverse problems; plays nicely with ipyparallel
- lmfit: https://lmfit.github.io/lmfit-py/model.html nicer fitting than scipy, horrible documentation
- shapely: https://shapely.readthedocs.io/en/latest/ Geometry
- sympy: http://www.sympy.org symbolic calculation
- sage: http://www.sagemath.org/ integrates sympy etc. for a full math software
- gtk/qt/tk: GUIs
suggested literature¶
- Python for Data Analysis, 2nd Edition (William McKinney) - German translation of 1st Edition available
- https://docs.python.org python tutorial (core only)
- https://scipy-lectures.org ("classic" numpy + scipy + matplotlib centered)