0010_fastcore_meta_summary

import

from fastdebug.utils import *
from fastdebug.core import *

fastcore and fastcore.meta

import fastcore

whichversion("fastcore")

fastcore: 1.5.27 
Python supercharged for fastai development    
Jeremy Howard and Sylvain Gugger 
https://github.com/fastai/fastcore/     
python_version: >=3.7     
/Users/Natsume/mambaforge/lib/python3.9/site-packages/fastcore

whatinside(fastcore, lib=True)

The library has 21 modules
['_modidx',
 '_nbdev',
 'all',
 'basics',
 'dispatch',
 'docments',
 'docscrape',
 'foundation',
 'imports',
 'meta',
 'nb_imports',
 'net',
 'parallel',
 'script',
 'shutil',
 'style',
 'test',
 'transform',
 'utils',
 'xdg',
 'xtras']

from fastcore.meta import *
import fastcore.meta as fm

What’s inside fastcore.meta

whatinside(fm, dun=True)

fastcore.meta has: 
13 items in its __all__, and 
43 user defined functions, 
19 classes or class objects, 
2 builtin funcs and methods, and
74 callables.

test_sig:            function    Test the signature of an object
FixSigMeta:          metaclass, type    A metaclass that fixes the signature on classes that override `__new__`
PrePostInitMeta:     metaclass, type    A metaclass that calls optional `__pre_init__` and `__post_init__` methods
AutoInit:            class, PrePostInitMeta    Same as `object`, but no need for subclasses to call `super().__init__`
NewChkMeta:          metaclass, type    Metaclass to avoid recreating object passed to constructor
BypassNewMeta:       metaclass, type    Metaclass: casts `x` to this class if it's of type `cls._bypass_type`
empty2none:          function    Replace `Parameter.empty` with `None`
anno_dict:           function    `__annotation__ dictionary with `empty` cast to `None`, returning empty if doesn't exist
use_kwargs_dict:     decorator, function    Decorator: replace `**kwargs` in signature with `names` params
use_kwargs:          decorator, function    Decorator: replace `**kwargs` in signature with `names` params
delegates:           decorator, function    Decorator: replace `**kwargs` in signature with params from `to`
method:              function    Mark `f` as a method
funcs_kwargs:        decorator, function    Replace methods in `cls._methods` with those from `kwargs`

Review individual funcs and classes

What is fastcore.meta all about?

It is a submodule contains 4 metaclasses, 1 class built by a metaclass, 4 decorators and a few functions.

Metaclasses give us the power to create new breeds of classes with new features.

Decorators give us the power to add new features to existing funcions.

We can find their basic info above

What can these metaclasses do for me?

We design/create classes to breed objects as we like.

We design/create metaclasses to breed classes as we like.

Before metaclasses, all classes are created by type and are born the same.

With metaclasses, e.g., FixSigMeta first uses type to its instance classes exactly like above, but then FixSigMeta immediately adds new features to them right before they are born.

FixSigMeta

can breed classes which are free of signature problems (or they can automatically fix signature problems).

PrePostInitMeta

inherited/evolved from FixSigMeta to breed classes which can initialize their objects using __pre_init__, __init__, __post_init__ whichever is available (allow me to abbreviate it as triple_init).

AutoInit

is an instance class created by PrePostInitMeta, and together with its own defined __pre_init__, subclasses of AutoInit has to worry about running super().__init__(...) no more.

• As AutoInit is an instance class created by PrePostInitMeta, it can pass on both features (free of signature problem and triple_init) to its subclasses.
• As it also defines its own __pre_init__ function which calls its superclass __init__ function, its subclasses will inherit this __pre_init__ function too.
• When subclasses of AutoInit create and initialize object intances through __call__ from PrePostInitMeta, AutoInit’s __pre_init__ runs super().__init__(...), so when we write __init__ function of subclasses which inherits from AutoInit, we don’t need to write super().__init__(...) any more.

NewChkMeta

is inherited from FixSigMeta, so any instance classes created by NewChkMeta can also pass on the no_signature_problem feature.

It defines its own __call__ to enable all the instance objects e.g., t created by all the instance classes e.g., T created by NewChkMeta to do the following:

• T(t) is t if isinstance(t, T) returns true
• when T(t) is t if not isinstance(t, T), or when T(t, 1) is t if isinstance(t, T) or when T(t, b=1) is t if isinstance(t, T), all return False

In other words, NewChkMeta creates a new breed of classes T as an example which won’t recreate the same instance object t twice. But if t is not T’s instance object, or we choose to add more flavor to t, then T(t) or T(t, 1) will create a new instance object of T.

BypassNewMeta

is inherited from FixSigMeta, so it has the feature of free from signature problems.

It defines its own __call__, so that when its instance classes _T create and initialize objects with a param t which is an instance object of another class _TestB, they can do the following:

• If _T likes _TestB and prefers t as it is, then when we run t2 = _T(t), and t2 is t will be True, and both are instances of _T.
  
• If _T is not please with t, it could be that _T does not like _TestB any more, then _T(t) is t will be False
• or maybe _T still likes _TestB, but want to add some flavors to t by _T(t, 1) or _T(t, b=1) for example, in this case _T(t) is t will also be False.

In other words, BypassNewMeta creates a new breed of instance classes _T which don’t need to create but make an object t of its own object instance, if t is an instance object of _TestB which is liked by _T and if _T likes t as it is.

What can those decorators do for me?

A decorator is a function that takes in a function and returns a modified function.

A decorator allows us to modify the behavior of a function.

use_kwargs_dict

allows us to replace an existing function’s param kwargs with a number of params with default values.

The params with their default values are provided in a dictionary.

use_kwargs

allows us to replace an existing function’s param kwargs with a number of params with None as their default values.

The params are provided as names in a list.

delegates

allows us to replace an existing function’s param kwargs with a number of params with their default values from another existing function.

In fact, delegates can work on function, classes, and methods.

funcs_kwargs

is a decorator to classes. It can help classes to bring in existing functions as their methods.

It can set the methods to use or not use self in the class.

The remaining functions

test_sig and method are straightforward, their docs tell it all clearly.

empty2none and anno_dict are no in use much at all. see the thread here.

fastview("FixSigMeta", nb=True)

class BaseMeta(FixSigMeta): 
    # using __new__ of  FixSigMeta instead of type
    def __call__(cls, *args, **kwargs): pass

class Foo_call_fix(metaclass=BaseMeta): # Base
    def __init__(self, d, e, f): pass

pprint(inspect._signature_from_callable(Foo_call_fix, sigcls=inspect.Signature))    

class FixSigMeta(type):===================================================================(0)       
    "A metaclass that fixes the signature on classes that override `__new__`"=============(1) # Any class having FixSigMeta as metaclass will have its own __init__ func stored in its attr __signature__;FixSigMeta uses its __new__ to create a class instance; then check whether its class instance has its own __init__;if so, remove self from the sig of __init__; then assign this new sig to __signature__ for the class instance;; 
    def __new__(cls, name, bases, dict):==================================================(2) # how does a metaclass create a class instance; what does super().__new__() do here;; 
        res = super().__new__(cls, name, bases, dict)=====================================(3)       
        if res.__init__ is not object.__init__: res.__signature__ = _rm_self(inspect.signature(res.__init__)) # how to remove self from a signature; how to check whether a class' __init__ is inherited from object or not;;  (4)
        return res========================================================================(5)       
                                                                                                                                                        (6)

Open 0003_Explore_document_FixSigMeta_PrePostInitMeta_AutoInit in Jupyter Notebook

ERROR:root:No traceback has been produced, nothing to debug.

fastsrcs()

test_sig.py
BypassNewMeta.py
snoop.py
FixSigMeta.py
fastnbs.py
funcs_kwargs.py
NewChkMeta.py
printtitle.py
AutoInit.py
method.py
_rm_self.py
delegates.py
create_explore_str.py
PrePostInitMeta.py
_funcs_kwargs.py
whatinside.py

fastview(FixSigMeta)

class BaseMeta(FixSigMeta): 
    # using __new__ of  FixSigMeta instead of type
    def __call__(cls, *args, **kwargs): pass

class Foo_call_fix(metaclass=BaseMeta): # Base
    def __init__(self, d, e, f): pass

pprint(inspect._signature_from_callable(Foo_call_fix, sigcls=inspect.Signature))    

class FixSigMeta(type):===================================================================(0)       
    "A metaclass that fixes the signature on classes that override `__new__`"=============(1) # Any class having FixSigMeta as metaclass will have its own __init__ func stored in its attr __signature__;FixSigMeta uses its __new__ to create a class instance; then check whether its class instance has its own __init__;if so, remove self from the sig of __init__; then assign this new sig to __signature__ for the class instance;; 
    def __new__(cls, name, bases, dict):==================================================(2) # how does a metaclass create a class instance; what does super().__new__() do here;; 
        res = super().__new__(cls, name, bases, dict)=====================================(3)       
        if res.__init__ is not object.__init__: res.__signature__ = _rm_self(inspect.signature(res.__init__)) # how to remove self from a signature; how to check whether a class' __init__ is inherited from object or not;;  (4)
        return res========================================================================(5)       
                                                                                                                                                        (6)

fastcodes("how to get signature's parameters", accu=0.8, nb=True, db=True)

keyword match is 0.8 , found a line: in delegates.py

        s2 = {k:v.replace(kind=inspect.Parameter.KEYWORD_ONLY) for k,v in inspect.signature(to_f).parameters.items() # How to access a signature's parameters as a dict?; How to replace the kind of a parameter with a different kind?; how to check whether a parameter has a default value?; How to check whether a string is in a dict and a list?; how dict.items() and dict.values() differ;  (14)

the entire source code in delegates.py

def low(a, b:int=1): pass
@delegates(low)
def mid(c, d:list=None, **kwargs): pass
pprint(inspect.signature(mid)) # pprint and inspect is loaded from fastdebug

def delegates(to:FunctionType=None, # Delegatee===========================================(0) # how to make delegates(to) to have to as FunctionType and default as None; 
              keep=False, # Keep `kwargs` in decorated function?==========================(1)       
              but:list=None): # Exclude these parameters from signature===================(2) # how to make delegates(to, but) to have 'but' as list and default as None; 
    "Decorator: replace `**kwargs` in signature with params from `to`"====================(3)       
    if but is None: but = []==============================================================(4)       
    def _f(f):============================================================================(5)       
        if to is None: to_f,from_f = f.__base__.__init__,f.__init__=======================(6) # how to write 2 ifs and elses in 2 lines; 
        else:          to_f,from_f = to.__init__ if isinstance(to,type) else to,f=========(7) # how to assign a,b together with if and else; 
        from_f = getattr(from_f,'__func__',from_f)========================================(8) # Is classmethod callable; does classmethod has __func__; can we do inspect.signature(clsmethod); how to use getattr(obj, attr, default); 
        to_f = getattr(to_f,'__func__',to_f)==============================================(9)       
        if hasattr(from_f,'__delwrap__'): return f========================================(10) # if B has __delwrap__, can we do delegates(A)(B) again?; hasattr(obj, '__delwrap__'); 
        sig = inspect.signature(from_f)===================================================(11) # how to get signature obj of B; what does a signature look like; what is the type; 
        sigd = dict(sig.parameters)=======================================================(12) # How to access parameters of a signature?; How to turn parameters into a dict?; 
        k = sigd.pop('kwargs')============================================================(13) # How to remove an item from a dict?; How to get the removed item from a dict?; How to add the removed item back to the dict?; when writing expressions, as they share environment, so they may affect the following code; 
        s2 = {k:v.replace(kind=inspect.Parameter.KEYWORD_ONLY) for k,v in inspect.signature(to_f).parameters.items() # How to access a signature's parameters as a dict?; How to replace the kind of a parameter with a different kind?; how to check whether a parameter has a default value?; How to check whether a string is in a dict and a list?; how dict.items() and dict.values() differ;  (14)
              if v.default != inspect.Parameter.empty and k not in sigd and k not in but}=(15)      
        anno = {k:v for k,v in getattr(to_f, "__annotations__", {}).items() if k not in sigd and k not in but} # How to get A's __annotations__?; How to access it as a dict?; How to select annotations of the right params with names?; How to put them into a dict?; How to do it all in a single line;  (16)
        sigd.update(s2)===================================================================(17) # How to add the selected params from A's signature to B's signature; How to add items into a dict;; 
        if keep: sigd['kwargs'] = k=======================================================(18) # How to add a new item into a dict;; 
        else: from_f.__delwrap__ = to_f===================================================(19) # How to create a new attr for a function or obj;; 
        from_f.__signature__ = sig.replace(parameters=sigd.values())======================(20) # How to update a signature with a new set of parameters;; 
        if hasattr(from_f, '__annotations__'): from_f.__annotations__.update(anno)========(21) # How to check whether a func has __annotations__; How add selected params' annotations from A to B's annotations;; 
        return f==========================================================================(22)      
    return _f=============================================================================(23)      
                                                                                                                                                        (24)

Open 0001_fastcore_meta_delegates in Jupyter Notebook

fastnotes("how make the most", folder="all")

keyword match is 1.0 , found a line: in 2022_part1/fastai-lecture-1.md

do you want to know how to make the most out of fastai? - do you know lesson 0 and the book meta learning by an alumni?

show 2 lines above and after in 2 0 2 2 _part1/fastai-lecture-1.md :

Do Jeremy and fastai community take it very seriously in help beginners along the way?

16:33 Make the most out of fast.ai

do you want to know how to make the most out of fastai? - do you know lesson 0 and the book meta learning by an alumni?

17:41 Learn in context

Do you know people learn naturally (better) with context rather than by theoretical curriculum? - Do you want this course to make you a competent deep learning practitioner by context and practical knowledge? - If you want theory from ground up, should you go to part 2 fastai 2019?

keyword match is 1.0 , found a line: in 2022_part1/fastai-lecture-2.md

• how to make the most out of fastai forum?

show 2 lines above and after in 2 0 2 2 _part1/fastai-lecture- 2 .md :

• Where can you get more quizzes of fastai and memorize them forever?

02:38 Introducing the forum

• how to make the most out of fastai forum?

04:12 Students’ works after week 1

06:08 A Wow moment

keyword match is 1.0 , found a line: in 2022_livecoding/live-coding-1.md

• • how to make the most out of .bash_history file?

show 2 lines above and after in 2 0 2 2 _livecoding/live-coding-1.md :

• • How to run jupyter lab without browser? jupyter lab --no-browser

• • How to find and check the content of .bash_history? cat .bash_history 1:12:53

• • how to make the most out of .bash_history file?

• • How to search the .bash_history commands? ctrl + r and use delete tab to clear the search, use esc tab to exit search

• • How to run a command starting with ju? !ju

What is fastcore.meta all about