Contents

Introduction V - Introduction to Python - I

Introduction V - Introduction to Python - I

Michael Ernst
Phd student - Fiebach Lab, Neurocognitive Psychology at Goethe-University Frankfurt

Before we get started…


Peer Herholz (he/him)
Research affiliate - NeuroDataScience lab at MNI/MIT
Member - BIDS, ReproNim, Brainhack, Neuromod, OHBM SEA-SIG, UNIQUE

logo logo   @peerherholz

Objectives 📍

  • learn basic and efficient usage of the python programming language

    • what is python & how to utilize it

    • building blocks of & operations in python

What is Python?

  • Python is a programming language

  • Specifically, it’s a widely used/very flexible, high-level, general-purpose, dynamic programming language

  • That’s a mouthful! Let’s explore each of these points in more detail…

Widely-used

  • Python is the fastest-growing major programming language

  • Top 3 overall (with JavaScript, Java)

The incredible growth of python, by David Robinson, 6.09.2017

High-level

Python features a high level of abstraction

  • Many operations that are explicit in lower-level languages (e.g., C/C++) are implicit in Python

  • E.g., memory allocation, garbage collection, etc.

  • Python lets you write code faster

File reading in Java

import java.io.BufferedReader;
import java.io.FileReader;
import java.io.IOException;
 
public class ReadFile {
    public static void main(String[] args) throws IOException{
        String fileContents = readEntireFile("./foo.txt");
    }
 
    private static String readEntireFile(String filename) throws IOException {
        FileReader in = new FileReader(filename);
        StringBuilder contents = new StringBuilder();
        char[] buffer = new char[4096];
        int read = 0;
        do {
            contents.append(buffer, 0, read);
            read = in.read(buffer);
        } while (read >= 0);
        return contents.toString();
    }
}

File-reading in Python

open(filename).read()

General-purpose

You can do almost everything in Python

  • Comprehensive standard library

  • Enormous ecosystem of third-party packages

  • Widely used in many areas of software development (web, dev-ops, data science, etc.)

Dynamic

Code is interpreted at [runtime](https://en.wikipedia.org/wiki/Runtime_(program_lifecycle_phase)

  • No compilation process*; code is read line-by-line when executed

  • Eliminates delays between development and execution

  • The downside: poorer performance compared to compiled languages

(Try typing import antigravity into a new cell and running it!)

What we will do in this section of the course is a short introduction to Python to help beginners to get familiar with this programming language.

It is divided into the following chapters:

Here’s what we will focus on in the first block:

Modules

Most of the functionality in Python is provided by modules. To use a module in a Python program it first has to be imported. A module can be imported using the import statement.

For example, to import the module math, which contains many standard mathematical functions, we can do:

import math

This includes the whole module and makes it available for use later in the program. For example, we can do:

import math

x = math.cos(2 * math.pi)

print(x)

1.0

where math.cos() is what is understood as a function (i.e a block of code which runs only when it is called). Functions can usually be identified by the Brackets () directly following the function name.

Importing the whole module is often times unnecessary and can lead to longer loading times or increased memory consumption. As an alternative to the previous method, we can also choose to import only a few selected functions from a module by explicitly listing which ones we want to import:

from math import cos, pi

x = cos(2 * pi)

print(x)

1.0

You can make use of tab again to get a list of functions/classes/etc. for a given module. Try it out via navigating the cursor behind the import statement and press tab:

from math import 

  Cell In [4], line 1
    from math import
                     ^
SyntaxError: invalid syntax

Comparably you can also use the help function (one of a numer of pythons build-in functions) to find out more about a given module:

import math
help(math)

It is also possible to give an imported module or symbol your own access name with the as additional:

import numpy as np
from math import pi as number_pi

x  = np.rad2deg(number_pi)

print(x)

You can provide any name (given it’s following python/coding conventions) but focusing on intelligibility won’t be the worst idea.

Remember “Readibility counts”

import matplotlib as pineapple

pineapple.

Exercise 1.1

Import the max from numpy and find out what it does.

note: the # character tells python not to run the following code in that line. It is therefore used to embedd comments in our code while avoiding errors. You can try deleting the # and see what happens.

# write your solution in this code cell

Exercise 1.2

Import the scipy package and assign the access name middle_earth and check its functions.

# write your solution in this code cell

Exercise 1.3

What happens when we try to import a module that is either misspelled or doesn’t exist in our environment or at all?

  1. python provides us a hint that the module name might be misspelled

  2. we’ll get an error telling us that the module doesn’t exist

  3. python automatically searches for the module and if it exists downloads/installs it

import welovethiscourse

A module consists of a python file (.py) containing code that will automatically run on import, but also allows us to access functions specfied in the module file. In the Introduction folder you’ll find a file called hello_statement.py

import hello_statement

Let’s have a look at the functions contained

help(hello_statement)

and call them as usual

hello_statement.some_func()

hello_statement.some_other_func()

hello_statement.some_other_other_func()

Namespaces and imports

  • Python is very serious about maintaining orderly namespaces (i.e unique identifiers)

  • If you want to use some code outside the current scope, you need to explicitly “import” it

  • Python’s import system often annoys beginners, but it substantially increases code clarity

    • Almost completely eliminates naming conflicts and confusion

Help and Descriptions

Using the function help we can get a description of almost all functions.

help(math.log)

math.log(10)

math.log(10, 2)

Variables and data types

  • in programming variables are things that store values

  • in Python, we declare a variable by assigning it a value with the = sign

    • name = value

    • code variables != math variables

      • in mathematics = refers to equality (statement of truth), e.g. y = 10x + 2

      • in coding = refers to assignments, e.g. x = x + 1 (instead we use ==to test for equality)

    • Variables are pointers, not data stores!

  • Python supports a variety of data types and structures:

    • booleans

    • numbers (ints, floats, etc.)

    • strings

    • lists

    • dictionaries

    • many others!

  • We don’t specify a variable’s type at assignment

Variables and types

Symbol names

Variable names in Python can contain alphanumerical characters a-z, A-Z, 0-9 and some special characters such as _. Normal variable names must start with a letter.

By convention, variable names start with a lower-case letter, and Class names start with a capital letter.

In addition, there are a number of Python keywords that cannot be used as variable names. These keywords are:

and, as, assert, break, class, continue, def, del, elif, else, except, exec, finally, for, from, global, if, import, in, is, lambda, not, or, pass, print, raise, return, try, while, with, yield

Assignment

(Not your homework assignment but the operator in python.)

The assignment operator in Python is =. Python is a dynamically typed language, so we do not need to specify the type of a variable when we create one.

Assigning a value to a new variable creates the variable:

# variable assignment
x = 1.0

Again, this does not mean that x equals 1 but that the variable x contains the value 1. Thus, our variable x is stored in the respective namespace:

x

1.0

This means that we can directly utilize the value of our variable:

x + 3

4.0

Although not explicitly specified, a variable does have a type associated with it. The type is derived from the value it was assigned.

type(x)

float

If we assign a new value to a variable, its type can change.

x = 1

type(x)

int

This outlines one more very important characteristic of python (and many other programming languages):

variables can be directly overwritten by assigning them a new value.

We don’t get an error like “This namespace is already taken.” Thus, always remember/keep track of what namespaces were already used to avoid unintentional deletions/errors (reproducibility/replicability much?).

ring_bearer = 'Bilbo'
ring_bearer

'Bilbo'

ring_bearer = 'Frodo'
ring_bearer

'Frodo'

If we try to use a variable that has not yet been defined we get an NameError

fellowship

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In [12], line 1
----> 1 fellowship

NameError: name 'fellowship' is not defined

Note for later sessions, that we will use in the notebooks try/except blocks to handle the exception, so the notebook doesn’t stop. The code below will try to execute the print function and if the NameError occurs the error message will be printed. Otherwise, an error will be raised. You will learn more about exception handling later.

try:
    print(Peeeeeer)
except(NameError) as err:
    print("NameError", err)
else:
    raise

NameError name 'Peeeeeer' is not defined

Variable names:

  • Can include letters (A-Z), digits (0-9), and underscores ( _ )

  • Cannot start with a digit

  • Are case sensitive (question: where did “lower/upper case” originate?)

This means that, for example:

  • shire0 is a valid variable name, whereas 0shire is not

  • shire and Shire are different variables

Exercise 2.1

Create the following variables n_elves, n_dwarfs, n_humans with the respective values 3, 7.0 and nine.

# write your solution here

Exercise 2.2

What’s the output of n_elves + n_dwarfs?

  1. n_elves + n_dwarfs

  2. 10

  3. 10.0

Exercise 2.3

Consider the following lines of code.

ring_bearer = 'Gollum'
ring_bearer
ring_bearer = 'Bilbo'
ring_bearer

What is the final output?

  1. 'Bilbo'

  2. 'Gollum'

  3. neither, the variable got deleted

Fundamental types & data structures

  • Most code requires more complex structures built out of basic data types

  • data type refers to the value that is assigned to a variable

  • Python provides built-in support for many common structures

    • Many additional structures can be found in the collections module

Most of the time you’ll encounter the following data types

  • integers (e.g. 1, 42, 180)

  • floating-point numbers (e.g. 1.0, 42.42, 180.90)

  • strings (e.g. "Rivendell", "Weathertop")

  • Boolean (True, False)

If you’re unsure about the data type of a given variable, you can always use the type() command.

Integers

Lets check out the different data types in more detail, starting with integers. Intergers are natural numbers that can be signed (e.g. 1, 42, 180, -1, -42, -180).

x = 1
type(x)

int

n_nazgul = 9
type(n_nazgul)

int

remaining_rings = -1
type(remaining_rings)

int

Floating-point numbers

So what’s the difference to floating-point numbers? Floating-point numbers are decimal-point numbers that can be signed (e.g. 1.0, 42.42, 180.90, -1.0, -42.42, -180.90).

x_float = 1.0
type(x_float)

float

n_nazgul_float = 9.0
type(n_nazgul_float)

float

remaining_rings_float = -1.0
type(remaining_rings_float)

float

Strings

Next up: strings.

Strings are basically text elements, from letters to words to sentences all are encoded as strings in python. In order to define a string, Python needs quotation marks, more precisely strings start and end with quotation marks, e.g. "Rivendell". You can choose between " and ' as both will work (NB: python will put ' around strings even if you specified "). However, it is recommended to decide on one and be consistent.

location = "Weathertop"
type(location)

str

abbreviation = 'LOTR'
type(abbreviation)

str

book_one = "The fellowship of the ring"
type(book_one)

str

Booleans

How about some Booleans? At this point it gets a bit more “abstract”. While there are many possible numbers and strings, a Boolean can only have one of two values: True or False. That is, a Boolean says something about whether something is the case or not. It’s easier to understand with some examples. First try the type() function with a Boolean as an argument.

True

True

b1 = True
type(b1)

bool

b2 = False
type(b2)

bool

lotr_is_awesome = True
type(lotr_is_awesome)

bool

Interestingly, True and False also have numeric values! True has an integer value of 1 and False has a value of 0.

True + True

2

wrongs = False + False
print(wrongs)
type(wrongs)

0

int

Converting data types

As mentioned before the data type is not set when assigning a value to a variable but determined based on its properties. Additionally, the data type of a given value can also be changed via set of functions.

  • int() -> convert the value of a variable to an integer

  • float() -> convert the value of a variable to a floating-point number

  • str() -> convert the value of a variable to a string

  • bool() -> convert the value of a variable to a Boolean

int("4")

4

float(3)

3.0

str(2)

'2'

bool(1)

True
Exercise 3.1

Define the following variables with the respective values and data types: fellowship_n_humans with a value of two as a float, fellowship_n_hobbits with a value of four as a string and fellowship_n_elves with a value of one as an integer.

# write your solution here
Exercise 3.2

What outcome would you expect based on the following lines of code?

  1. True - False

  2. type(True)

# write your solution here
Exercise 3.3

Define two variables, fellowship_n_dwarfs with a value of one as a string and fellowship_n_wizards with a value of one as a float.

Subsequently, change the data type of fellowship_n_dwarfs to integer and the data type of fellowship_n_wizard to string.

# write your solution here

# write your solution here

Why do programming/science in Python?

Lets go through some advantages of the python programming language.


https://funvizeo.com/media/memes/9114fb92b16ca1b8/java-python-think-why-waste-time-word-when-few-word-trick-meme-7a08727102156f3c-e9db4e91c4b2a7d5.jpg

Easy to learn

  • Readable, explicit syntax

  • Most packages are very well documented

    • e.g., scikit-learn’s documentation is widely held up as a model

  • A huge number of tutorials, guides, and other educational materials

Comprehensive standard library

  • The Python standard library contains a huge number of high-quality modules

  • When in doubt, check the standard library first before you write your own tools!

  • For example:

    • os: operating system tools

    • re: regular expressions

    • collections: useful data structures

    • multiprocessing: simple parallelization tools

    • pickle: serialization

    • json: reading and writing JSON

Exceptional external libraries

  • Python has very good (often best-in-class) external packages for almost everything

  • Particularly important for “data science”, which draws on a very broad toolkit

  • Package management is easy (conda, pip)

  • Examples:

    • Web development: flask, Django

    • Database ORMs: SQLAlchemy, Django ORM (w/ adapters for all major DBs)

    • Scraping/parsing text/markup: beautifulsoup, scrapy

    • Natural language processing (NLP): nltk, gensim, textblob

    • Numerical computation and data analysis: numpy, scipy, pandas, xarray, statsmodels, pingouin

    • Machine learning: scikit-learn, Tensorflow, keras

    • Image processing: pillow, scikit-image, OpenCV

    • audio processing: librosa, pyaudio

    • Plotting: matplotlib, seaborn, altair, ggplot, Bokeh

    • GUI development: pyQT, wxPython

    • Testing: py.test

    • Etc. etc. etc.

(Relatively) good performance

  • Python is a high-level dynamic language — this comes at a performance cost

  • For many (not all!) use cases, performance is irrelevant most of the time

  • In general, the less Python code you write yourself, the better your performance will be

    • Much of the standard library consists of Python interfaces to C functions

    • Numpy, scikit-learn, etc. all rely heavily on C/C++ or Fortran

Python vs. other data science languages

  • Python competes for mind share with many other languages

  • Most notably, R

  • To a lesser extent, Matlab, Mathematica, SAS, Julia, Java, Scala, etc.

R

  • R is wide-spread in traditional statistics and some fields of science

    • Has attracted many SAS, SPSS, and Stata users

  • Exceptional statistics support; hundreds of best-in-class libraries

  • Designed to make data analysis and visualization as easy as possible

  • Slow

  • Language quirks drive many experienced software developers crazy

  • Less support for most things non-data-related

MATLAB

  • A proprietary numerical computing language widely used by engineers

  • Good performance and very active development, but expensive

  • Closed ecosystem, relatively few third-party libraries

    • There is an open-source port (Octave)

  • Not suitable for use as a general-purpose language

So, why Python?

Why choose Python over other languages?

  • Arguably none of these offers the same combination of readability, flexibility, libraries, and performance

  • Python is sometimes described as “the second best language for everything”

  • Doesn’t mean you should always use Python

    • Depends on your needs, community, etc.

via GIPHY

You can have your cake and eat it!

  • Many languages—particularly R—now interface seamlessly with Python (r2py, Rserve, and pyRserve)

  • You can work primarily in Python, fall back on R when you need it (or vice versa)

  • The best of all possible worlds?

The core Python “data science” stack

  • The Python ecosystem contains tens of thousands of packages

  • Several are very widely used in data science applications:

  • We’ll cover the first three very briefly here

    • Other tutorials will go into greater detail on most of the others

The core “Python for psychology” stack

  • The Python ecosystem contains tens of thousands of packages

  • Several are very widely used in psychology research:

    • Jupyter: interactive notebooks

    • Numpy: numerical computing in Python

    • pandas: data structures for Python

    • Scipy: scientific Python tools

    • Matplotlib: plotting in Python

    • seaborn: plotting in Python

    • scikit-learn: machine learning in Python

    • statsmodels: statistical analyses in Python

    • pingouin: statistical analyses in Python

    • psychopy: running experiments in Python

    • nilearn: brain imaging analyses in `Python``

    • mne: electrophysiology analyses in Python

  • Execept scikit-learn, nilearn and mne, we’ll cover all very briefly in this course

    • there are many free tutorials online that will go into greater detail and also cover the other packages

Homework assignment #3

Your third homework assignment will entail working through a few tasks covering the contents discussed in this session within of a jupyter notebook. You can download ithere. In order to open it, put the homework assignment notebook within the folder you stored the course materials, start a jupyter notebook as during the sessions, navigate to the homework assignment notebook, open it and have fun!

Deadline: 03/12/2021, 11:59 PM EST

previous

Introduction V

next

Introduction VI