Python Context Management Protocol
Filed in: Python | Context Management Protocol
In order to understand the context management protocol we need to look at the paradigm it is trying to solve. What the idea behind context management is to manage the context or resource being used. It is easy to forget to close a file after you have opened it and this can cause a memory leak as you have resources hogging memory you don’t need. Further more you can get unreliable results from not closing files. Wouldn’t it be nice if we could just use a file and not worry about closing it behind us ? Well there is a simple way to do this in Python. If you use the open command in combination with a with statement then once the statement bloc or code suite completes the file is auto closed for you.
The question then becomes exactly how does this happen in the background. Let us take this simple piece of code it opens a file and writes to it using the with statement. Note open supports the context manager protocol so this works.
fd.write('Howdy')
Now would it not be great if I could create a function that manages a resource and has decorated starting and ending functionality. Well this is achievable with the help of some magics and some Protocol convention. Let me explain what I mean by Magics first.
The magics are methods that are overridden from object that all have __someFunc__ so you know they are special functions so __init__ is a special as is __enter__and __exit__. All magics are prepended with 2 underscores and post pended with 2 underscores.
Now let me explain what the context manager protocol is. The protocol basically says if you have a class with __init__ and __entry__ and __exit__ you can happily use it in a “with” context and the with will call __init__ to set up the object and then __enter__ to start the processing and __exit__ when the code block ends.
If we were to replicate a simple file descriptor class that mimics the open(…) functionality used in “with” we could do this :
def __init__(self, filename, mode):
self.filename = filename
self.mode = mode
def __enter__(self):
self.fd = open(self.filename, self.mode)
return self.fd
def __exit__(self, *args):
# technically the *args hides __exit__(self, exception_type, exception_value, traceback): which allows you to catch exception details in case they are thrown.
self.fd.close()
with MyFile('test.txt', 'w') as fd:
fd.write('Howdy There ')
In [ ]:
You can see how we have a new context manager called MyFile that can be used in the “with” context because we support the required methods.
We further open our file and then when the context completes we close the file in __exit__ so we never have open files.
Now you know the basics we can expand the idea to look after any resource we have for example database connections, queue connections etc...
This is essentially the essence of the context management protocol which is the idea that we have some agreed flow (__init__->__entry__->our code using the resource->__exit__) if we use the “with” context and we implement the correct methods.
Python Decorator Semantics
Naive Bayes classification AI algorithm
K-Means Clustering AI algorithm
Equity Derivatives tutorial
Fixed Income tutorial
Java
python
Scala
Investment Banking tutorials
HOME
The question then becomes exactly how does this happen in the background. Let us take this simple piece of code it opens a file and writes to it using the with statement. Note open supports the context manager protocol so this works.
with open('test.txt', 'w') as fd:
fd.write('Howdy')
Now would it not be great if I could create a function that manages a resource and has decorated starting and ending functionality. Well this is achievable with the help of some magics and some Protocol convention. Let me explain what I mean by Magics first.
The magics are methods that are overridden from object that all have __someFunc__ so you know they are special functions so __init__ is a special as is __enter__and __exit__. All magics are prepended with 2 underscores and post pended with 2 underscores.
Now let me explain what the context manager protocol is. The protocol basically says if you have a class with __init__ and __entry__ and __exit__ you can happily use it in a “with” context and the with will call __init__ to set up the object and then __enter__ to start the processing and __exit__ when the code block ends.
If we were to replicate a simple file descriptor class that mimics the open(…) functionality used in “with” we could do this :
class MyFile():
def __init__(self, filename, mode):
self.filename = filename
self.mode = mode
def __enter__(self):
self.fd = open(self.filename, self.mode)
return self.fd
def __exit__(self, *args):
# technically the *args hides __exit__(self, exception_type, exception_value, traceback): which allows you to catch exception details in case they are thrown.
self.fd.close()
with MyFile('test.txt', 'w') as fd:
fd.write('Howdy There ')
In [ ]:
You can see how we have a new context manager called MyFile that can be used in the “with” context because we support the required methods.
We further open our file and then when the context completes we close the file in __exit__ so we never have open files.
Now you know the basics we can expand the idea to look after any resource we have for example database connections, queue connections etc...
This is essentially the essence of the context management protocol which is the idea that we have some agreed flow (__init__->__entry__->our code using the resource->__exit__) if we use the “with” context and we implement the correct methods.
People who enjoyed this article also enjoyed the following:
Python Decorator Semantics
Naive Bayes classification AI algorithm
K-Means Clustering AI algorithm
Equity Derivatives tutorial
Fixed Income tutorial
And the following Trails:
C++Java
python
Scala
Investment Banking tutorials
HOME