Python Advanced (2023-04-24 - 2023-04-26)

Original Agenda

• Agenda: Python Advanced (2023-04-24 - 2023-04-26)

Github Repository

• https://github.com/jfasch/2023-04-24

Notebook

• Structure: Jupyter Notebook (Structure): Python Advanced (2023-04-24 - 2023-04-26)

• Entire notebook: notebook.ipynb

Material

• Python Programming: From Absolute Beginner to Advanced Productivity

Day 1: Python Inside Out

Basics Recap

• Variables

• Datatypes

• Compound Datatypes

• Sequential Datatypes

• Indexing and Slicing

• References, (Im)mutability

• for Loops

• The range Function

Generators

From Iteration, Generators, And yield)

• The Iterator Protocol

• Implementing A Fibonacci Number Generator

Dynamic Features

• The exec() Function

• The eval() function

Object Oriented Programming

From Object Oriented Programming …

• Classes And Dictionaries

• Constructor

• Methods

Day 2: Object Oriented Programming, And Miscellaneous Discussion

Morning Opening

Company code review. Cool, clean. Performance hint: use generators a lot.

• Generate dataframes that appear in a directory from time to time: watch-dataframes.py

• Generate mass data which would otherwise have to be loaded into memory: iter-measurement.py

Object Oriented Programming (Continued)

From Object Oriented Programming …

• str() And repr()

• Properties

• collections.namedtuple

• Inheritance

• Operator Overloading

• Abstract Base Classes (abc), And Duck Typing

Clean Code?

From Is Software A Craft? Software Is A Craft! ⟶ Clean Code …

• What’s There? What Have We Accomplished?

• Clean Code: Messages From The Book

Group Project Kickoff

• Big picture

  

  • Data acquisition: gathering sensor data

  • Varying kinds of sensors: physical hardware, internet proxy, MQTT, …

  • Varying kinds of information sinks: CSV, database, MQTT, …

  • Acquisition machinery: timestamping, frequency, buffering, …

• Start hacking: “data logger device”, first sketch. To be implemented by three groups.

  • datalogger-joerg.py

Day 3: Group Project

Project Setup (IDE Usage?)

Apart from any kind of IDE (IDE’s tend to hide those details from you - this is not to help you, but to make you depend on them), these are the steps to make our project work:

• Project Setup

Related topics:

• Python Package Index

• Virtual Environments

How to make VS Code work (terminal is simpler, I’d say 🤨):

• Using Python environments in VS Code

Continue Hacking

Until 10am; explain:

• File I/O

  Someone said, “Lets check for file existence first, and create the file if it does not exist.” ⟶ Exclusive creation! (open(filename, 'x'))

• The unittest Module

  Someone was using unittest. Not exactly deprecated, but pytest (see pytest Introduction, By Example) rocks (not only compared to unittest, but absolutely)

• Random notes

  • Reopen file in every loop iteration (open(..., 'a'))? Better leave it open until done? ⟶ RAII jjj link

  • Polymorphize loop iterations? Infinite vs. finite?

  • KeyboardInterrupt? Termination of an infinite loop? Graceful termination? ⟶ threading in unit test (setting stop flag in main thread). Ugly!

Test Driven Development, pytest

• (Try to) establish mindset

• pytest Introduction, By Example

• pytest fixtures

  • How to use temporary directories and files in tests

  • How to monkeypatch/mock modules and environments

Untold

Closures, Decorators And More

• Positional and Keyword Arguments

• Closures

• Decorators

Error Handling And Recovery

• Exception Handling

• Context Managers: The with Statement

Miscellaneous Threading

The GIL

• GIL, over at the Python Wiki

• David Beazley: Understanding the Python GIL (Youtube)

• Show how the load/modify/store conflict appears to not be an issue in Python (?) (code/race-load-modify-store.py)

  • Could that be the GIL

  • What’s the GIL altogether?

  • Why could it be a bad idea to trust the GIL too much? (⟶ PEP 703)

  • Jason Brownlee mentions the same “issue” – there is no race condition. There used to, he says, before Python 3.10, but nowadays there’s not. He shows an contrived way to show a race (involving a time.sleep(0) 🙄)

  • Fix that with a mutex (show with usage!)

  • Events. Maybe some live-hacked dice-rolling in a subthread.

  • queue — A synchronized queue class

• Multiprocessing

  Simply walk through the introductory example in multiprocessing — Process-based parallelism

Design Patterns Overview

• UML Short Introduction

• OO Design Principles

• Design patterns: problems, possible solutions