path: root/research/python.md

[[!meta title="Python"]]

## Learning Python

### Which version to start? 2.x or 3.x?

Short answer: start learning 3.x and, if needed, check the differences with 2.x.

From [Should I use Python 2 or Python 3 for my development activity?](Should I use Python 2 or Python 3 for my development activity?):

    Besides, several aspects of the core language (such as print and exec being
    statements, integers using floor division) have been adjusted to be easier for
    newcomers to learn and to be more consistent with the rest of the language, and
    old cruft has been removed (for example, all classes are now new-style,
    "range()" returns a memory efficient iterable, not a list as in 2.x).

    [...]

    In particular, instructors introducing Python to new programmers should
    consider teaching Python 3 first and then introducing the differences in Python
    2 afterwards (if necessary), since Python 3 eliminates many quirks that can
    unnecessarily trip up beginning programmers trying to learn Python 2.

Also:

* [Python Future: Easy, clean, reliable Python 2/3 compatibility](http://python-future.org/).
* [Should I learn Python 2 or 3?](https://www.dataquest.io/blog/python-2-or-3/).

### General

* Everything is an object. Really? What about symbols like + - and =?
* The `dir()` and `help()` functions are really useful.
* Great idea: iteration protocol.
* There are sequences and sum operations common for all types and specific type operations.

### Iteration and optimization

    In general, leading and trailing double underscores is the naming pattern
    Python uses for implementation details. The names without the underscores in
    this list are the callable methods on string objects.

### Polymorphism

Python encourages polymorphism:

    This is related to the idea of polymorphism mentioned earlier, and it stems
    from Python’s lack of type declarations. As you’ll learn, in Python, we code to
    object interfaces (operations supported), not to types. That is, we care what
    an object does, not what it is. Not caring about specific types means that code
    is automatically applicable to many of them—any object with a compatible
    interface will work, regardless of its specific type. Although type checking is
    supported—and even required in some rare cases—you’ll see that it’s not usually
    the “Pythonic” way of thinking. In fact, you’ll find that polymorphism is
    probably the key idea behind using Python well.

### Numeric Display Formats

* [14. Floating Point Arithmetic: Issues and Limitations — Python 2.7.13 documentation](https://docs.python.org/2/tutorial/floatingpoint.html)
* [What Every Computer Scientist Should Know About Floating-Point Arithmetic](https://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html)
* [Floating-point arithmetic - Wikipedia](https://en.wikipedia.org/wiki/Floating-point_arithmetic).

    This floating-point limitation is especially apparent for values that cannot be
    represented accurately given their limited number of bits in memory.

    [...]

    fractions and decimals both allow more intuitive and accurate results than
    floating points sometimes can, in different ways—by using rational
    representation and by limiting precision

### Types

    More formally, there are three major type (and operation) categories in Python
    that have this generic nature:

    Numbers (integer, floating-point, decimal, fraction, others)
    Support addition, multiplication, etc.

    Sequences (strings, lists, tuples)
    Support indexing, slicing, concatenation, etc.

    Mappings (dictionaries)
    Support indexing by key, etc.

    [...]

    The major core types in Python break down as follows:

    Immutables (numbers, strings, tuples, frozensets)
    None of the object types in the immutable category support in-place changes,
    though we can always run expressions to make new objects and assign their
    results to variables as needed.

    Mutables (lists, dictionaries, sets, bytearray)
    Conversely, the mutable types can always be changed in place with operations
    that do not create new objects. Although such objects can be copied, in-place
    changes support direct modification.

So remember that when [copying](https://stackoverflow.com/questions/2612802/how-to-clone-or-copy-a-list#2612815)
or referencing a list.

Also, [take care with handling mutables as arguments and as default arguments](http://www.thedigitalcatonline.com/blog/2015/02/11/default-arguments-in-python/),
also explained [here](https://docs.python.org/3/tutorial/controlflow.html#default-argument-values) and [here](https://docs.python-guide.org/writing/gotchas/)
(common gotchas).

From [Scopes an Namespaces](https://docs.python.org/3/tutorial/classes.html#python-scopes-and-namespaces), telling that assignments bind names to objects:

    A special quirk of Python is that – if no global statement is in effect –
    assignments to names always go into the innermost scope. Assignments do not
    copy data — they just bind names to objects. The same is true for deletions:
    the statement del x removes the binding of x from the namespace referenced by
    the local scope. In fact, all operations that introduce new names use the local
    scope: in particular, import statements and function definitions bind the
    module or function name in the local scope.

    The global statement can be used to indicate that particular variables live in
    the global scope and should be rebound there; the nonlocal statement indicates
    that particular variables live in an enclosing scope and should be rebound
    there.

    [...]

    Actually, you may have guessed the answer: the special thing about methods is
    that the instance object is passed as the first argument of the function. In
    our example, the call x.f() is exactly equivalent to MyClass.f(x). In general,
    calling a method with a list of n arguments is equivalent to calling the
    corresponding function with an argument list that is created by inserting the
    method’s instance object before the first argument.

Week references (from [here](https://docs.python.org/3/tutorial/stdlib2.html):

    Python does automatic memory management (reference counting for most objects
    and garbage collection to eliminate cycles). The memory is freed shortly after
    the last reference to it has been eliminated.

Now explain this:

    Python 2.7.13 (default, Sep 26 2018, 18:42:22)
    [GCC 6.3.0 20170516] on linux2
    Type "help", "copyright", "credits" or "license" for more information.
    >>> hex(id([]))
    '0x7f6264bbf368'
    >>> hex(id([]))
    '0x7f6264bbf368'
    >>> x = []
    >>> hex(id(x))
    '0x7f6264bbf368'     # both x and [] points to the same memory location
    >>> x.append('0')
    >>> hex(id(x))
    '0x7f6264bbf368'     # x still points to the same memory location
    >>> hex(id([]))
    '0x7f6264baeab8'     # now [] points somewhere else
    >>> hex(id('test'))
    '0x7f6264bc9480'
    >>> x = 'test'
    >>> hex(id(x))
    '0x7f6264bc9450'
    >>> hex(id('test'))
    '0x7f6264bc9450'
    >>> hex(id('another test'))
    '0x7f6264bcc1f0'
    >>> x = 'another test'
    >>> hex(id(x))
    '0x7f6264bcc228'
    >>> hex(id('another test'))
    '0x7f6264bcc260'
    >>> 

### Threads

From [GlobalInterpreterLock](https://wiki.python.org/moin/GlobalInterpreterLock):

    In CPython, the global interpreter lock, or GIL, is a mutex that protects
    access to Python objects, preventing multiple threads from executing Python
    bytecodes at once. This lock is necessary mainly because CPython's memory
    management is not thread-safe. (However, since the GIL exists, other features
    have grown to depend on the guarantees that it enforces.)

    [...]

    The GIL is controversial because it prevents multithreaded CPython programs
    from taking full advantage of multiprocessor systems in certain situations.
    Note that potentially blocking or long-running operations, such as I/O, image
    processing, and NumPy number crunching, happen outside the GIL. Therefore it is
    only in multithreaded programs that spend a lot of time inside the GIL,
    interpreting CPython bytecode, that the GIL becomes a bottleneck. 

From: [Thread State and the Global Interpreter Lock](https://docs.python.org/3/c-api/init.html#thread-state-and-the-global-interpreter-lock):

    When threads are created using the dedicated Python APIs (such as the threading
    module), a thread state is automatically associated to them and the code showed
    above is therefore correct. However, when threads are created from C (for
    example by a third-party library with its own thread management), they don’t
    hold the GIL, nor is there a thread state structure for them.

### Nice stuff

* [Verbose Regular Expressions](http://www.diveintopython3.net/regular-expressions.html#verbosere).

## Implementations

* [MicroPython - Python for microcontrollers](http://micropython.org/) ([compiling](https://github.com/micropython/micropython/wiki/Getting-Started).

## Libraries and applications

* QGIS.
* [SciPy.org — SciPy.org](https://www.scipy.org/) ([package](https://packages.debian.org/stable/python-scipy)).

## Frameworks

* [Welcome | Flask (A Python Microframework)](http://flask.pocoo.org/) ([package](https://packages.debian.org/stretch/python-flask)).
* Async: [asyncio](https://docs.python.org/3/library/asyncio.html), Twisted and Tornado.
* [Anaconda Data Science Platform](https://www.anaconda.com/).

## IDEs

* [PyCharm](https://www.jetbrains.com/pycharm/).

## Misc

* [Indentation](https://www.python.org/dev/peps/pep-0008/#indentation): Use 4 spaces per indentation level.

## Test projects

* [Arduino Blog » How close are we to doomsday? A clock is calculating it in real time](https://blog.arduino.cc/2013/03/27/how-close-are-we-to-doomsday-clock/) ([python code](https://github.com/tomschofield/Neurotic-Armageddon-Indicator/blob/master/NAI_SERVER/nai_scraper.py) to parse [Timeline from the Bulletin of the Atomic Scientists](http://thebulletin.org/timeline)).