Agenda: Python Advanced (2023-10-24 - 2023-10-26)#

Up Front: Project Management#

• Windows Subsystem for Linux (WSL)

• Python Package Index

• Virtual Environments

Exercise Prerequisites#

• Commandline Arguments (sys.argv)

• Commandline Parsing with argparse

Datatypes, Datatypes#

• Variables

• Datatypes

• Compound Datatypes

• Sequential Datatypes

• Indexing and Slicing

• The range Function

• More About Strings

• String Formatting

• Miscellaneous String Methods

• More About Lists

• More About Dictionaries

• More About Sets

• File I/O

• References, (Im)mutability

Functions#

• Functions (Slideshow)

• Functions: Global and Local Scope (Livehacking Screenplay)

Looping Constructs (With An Emphasis On The else Clause)#

• while Loops

• for Loops

Generators#

• The Iterator Protocol

• Implementing A Fibonacci Number Generator

Miscellaneous#

• Comprehensions (List, Dictionary, Set)

Closures, Decorators And More#

• Positional and Keyword Arguments

• Closures

• Decorators

Dynamic Features#

• The exec() Function

• The eval() function

Object Oriented Programming#

From Object Oriented Programming …

• Classes And Dictionaries

• Constructor

• Methods

• str() And repr()

• Properties

• collections.namedtuple

• Inheritance

• Operator Overloading

• Abstract Base Classes (abc), And Duck Typing

Error Handling And Recovery#

• Exception Handling

• Context Managers: The with Statement

Modules And Packages#

• Modules and Packages

Multithreading#

• Multithreading (Python)

• Race Conditions, and Mutexes

• Discussion: The Global Interpreter Lock (GIL)

  • Show how the load/modify/store conflict appears to not be an issue in Python (?) (code/race-load-modify-store.py)

  • Could that be the GIL ?

  • What’s the GIL altogether?

  • Why could it be a bad idea to trust the GIL too much? (⟶ PEP 703)

  • Jason Brownlee mentions the same “issue” – there is no race condition. There used to, he says, before Python 3.10, but nowadays there’s not. He shows an contrived way to show a race (involving a time.sleep(0) 🙄)

  • Fix that with a mutex (show with usage!)

• queue — A synchronized queue class

Multiprocessing#

Simply walk through the introductory example in multiprocessing — Process-based parallelism

Data Science Topics#

• CSV Files

• Databases

• Pandas

• From Machine Learning, Artificial Intelligence

  • Machine Learning: Concepts and Terminology

  • Linear Regression

  • K-Means

Test Driven Development#

• Establish mindset

• pytest Introduction, By Example

• pytest fixtures

  • How to use temporary directories and files in tests

  • How to monkeypatch/mock modules and environments

Group Project: An Idea#

• Data acquisition: gathering sensor data

• Varying kinds of sensors: physical hardware, internet proxy, MQTT, …

• Varying kinds of information sinks: CSV, database, MQTT, …

• Acquisition machinery: timestamping, frequency, buffering, …