Using Django Lock Manager¶
This is the main module file for Django Lock Manager (django-lockmgr) and contains lock management functions/classes.
There are two ways you can use Django Lock Manager:
The first (and recommended) way, is to use the context manager class
LockMgr
.The second (lower level) way, is to use the lock functions directly, such as
get_lock()
,unlock()
, andset_lock()
.
Using the context manager LockMgr (recommended)¶
LockMgr
is a wrapper class for the various locking functions in this module, e.g. get_lock()
, and
is designed to be used as a context manager, i.e. using a with
statement.
It’s strongly recommended to use django-lockmgr via the LockMgr
context manager unless you have a specific
need for manual lock management, as it greatly reduces the risk of “stuck locks” due to human error, or
incorrect exception handling.
By using django-lockmgr via this context manager, it ensures you don’t forget to release any locks after you’ve finished with the resources you were using.
Not only that, but it also ensures in the event of an exception, or an unexpected crash of your application,
that your locks will usually be safely released by LockMgr.__exit__()
.
>>> from lockmgr.lockmgr import LockMgr
>>> try:
... with LockMgr('mylock', 60) as l:
... print('Doing stuff with mylock locked.')
... # Obtain an additional lock for 'otherlock' - will use the same expiry as mylock
... # Since ``ret`` is set to True, it will return a bool instead of raising Lock
... if l.lock('otherlock', ret=True):
... print('Now otherlock is locked...')
... l.unlock('otherlock')
... else:
... print('Not doing stuff because otherlock is already locked...')
... # If you're getting close to your lock's expiry (timeout), you can call '.renew()' to add an extra
... # 2 minutes to your expiry time. Or manually specify the expiry with 'expires=120'
... sleep(50)
... l.renew(expires=30) # Add an extra 30 seconds to the expiration of 'mylock'
... except Locked as e:
... print('Failed to lock. Reason: ', type(e), str(e))
Using the raw module lock management functions¶
In some cases, it might not be suitable to use context management due to a complex application flow, such as the use of threading / multiprocessing, sharing the locks across other applications, etc.
If you need to, you can access the lower level lock management functions by importing this module, or the individual functions.
Here’s some examples:
First, let’s get a lock using get_lock()
that expires in 10 seconds, and wait a few seconds.
>>> from lockmgr import lockmgr
>>> lk = lockmgr.get_lock('my_app:somelock', expires=10)
>>> sleep(5)
Since our lock is going to expire soon, we’ll use renew_lock()
to reset the expiration time to 20 seconds
from now.
>>> lk = lockmgr.renew_lock(lk, 20) # Change the expiry time to 20 seconds from now
>>> sleep(15)
Using is_locked()
, we can confirm that the lock ``my_app:somelock` is still locked:
>>> lockmgr.is_locked('my_app:somelock') # 15 seconds later, the lock is still locked
True
Finally, we use unlock()
to release the lock. You can pass either a string lock name such as
my_app:somelock
, or you can also pass a Lock
database object i.e. the result from get_lock()
.
Use whichever parameter type you prefer, it doesn’t make a difference.
>>> lockmgr.unlock(lk)
Extra documentation¶
This is not the end of the documentation, this is only the beginning! :)
You’ll find detailed documentation on the pages for each function / class / method. Most things are documented
using PyDoc, which means you can view usage information straight from most Python IDEs (e.g. PyCharm and VS Code),
as well as via the help()
function inside of the Python REPL.
Browsable HTML API docs
We have online documentation for this module, which shows the usage information for each individual function and class method in this module.
Python REPL help
Using the help()
function, you can view help on modules, classes, functions and more straight from the REPL:
$ ./manage.py shell
Python 3.8.0 (v3.8.0:fa919fdf25, Oct 14 2019, 10:23:27)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> from lockmgr import lockmgr
>>> help(lockmgr.get_lock)
Below is a screenshot showing the REPL help page for get_lock()
-
exception
lockmgr.lockmgr.
LockFail
(*args, lock: lockmgr.models.Lock = None)[source]¶ Raised when locks were requested, with failure/rollback if any already existed.
-
class
lockmgr.lockmgr.
LockMgr
(name, expires: Optional[int] = 600, locked_by=None, lock_process=None, wait: int = None)[source]¶ LockMgr is a wrapper class for the various locking functions in this module, e.g.
get_lock()
, and is designed to be used as a context manager, i.e. using awith
statement.By using django-lockmgr via this context manager, it ensures you don’t forget to release any locks after you’ve finished with the resources you were using.
Not only that, but it also ensures in the event of an exception, or an unexpected crash of your application, that your locks will usually be safely released by
__exit__()
.Usage:
Using a
with
statement, create a LockMgr formylock
with automatic expiration if held for more than 60 seconds. After thewith
statement is completed, all locks created will be removed.>>> try: ... with LockMgr('mylock', 60) as l: ... print('Doing stuff with mylock locked.') ... # Obtain an additional lock for 'otherlock' - will use the same expiry as mylock ... # Since ``ret`` is set to True, it will return a bool instead of raising Lock ... if l.lock('otherlock', ret=True): ... print('Now otherlock is locked...') ... l.unlock('otherlock') ... else: ... print('Not doing stuff because otherlock is already locked...') ... except Locked as e: ... print('Failed to lock. Reason: ', type(e), str(e))
You can also use
renew()
to request more time / re-create the lock if you’re close to, or have already exceeded the lock expiration time (defaults to 10 mins).>>> try: ... with LockMgr('mylock', 60) as l: ... print('Doing stuff with mylock locked.') ... sleep(50) ... l.renew(expires=30) # Add an additional 30 seconds of time to the lock expiration ... sleep(50) # It's now been 100 seconds. 'mylock' should be expired. ... # We can still renew an expired lock when using LockMgr. It will simply re-create the lock. ... l.renew() # Add an additional 120 seconds (default) of time to the lock expiration ... except Locked as e: ... print('Failed to lock. Reason: ', type(e), str(e))
-
expires
= None¶ The user supplied expiration time in seconds
-
lock
(name, expires: int = None, ret: bool = False, wait: int = None)[source]¶ Obtains a lock using
get_lock()
and appends it to_locks
if successful.If the argument
ret
isFalse
(default), it will raiseLocked
if the lock couldn’t be obtained.Otherwise, if
ret
isTrue
, it will simply returnFalse
if the requested lock name is already locked.- Parameters
name (str) – A unique name to identify your lock
expires (int) – (Default: 600 sec) How long before this lock is considered stale and forcefully released?
ret (bool) – (Default: False) Return
False
if locked, instead of raisingLocked
.wait (int) – (Optional) Retry obtaining the lock for this many seconds. MUST be divisible by 5. If not empty, will retry obtaining the lock every 5 seconds until
wait
seconds
- Raises
Locked – If the requested lock
name
is already locked elsewhere,Locked
will be raised- Return bool success
True
if successful. Ifret
is true then will also return False on failure.
-
lock_process
= None¶ Usually None, but sometimes may represent the process ID this lock belongs to
-
locked_by
= None¶ Who/what created this lock - usually the hostname unless manually specified
-
name
= None¶ The lock name (from the constructor)
-
renew
(lock: Union[str, lockmgr.models.Lock] = None, expires: int = 120, add_time: bool = True, **kwargs) → lockmgr.models.Lock[source]¶ Add
expires
seconds to the lock expiry time oflock
. Iflock
isn’t specified, will default to the class instance’s original lockmain_lock
Alias for
renew_lock()
- but withadd_time
andcreate
set toTrue
by default, instead ofFalse
.With no arguments specified, this method will renew the main lock of the class
main_lock
for an additional 2 minutes (or if the lock is already expired, will re-create it with 2 min expiry).Example usage:
>>> with LockMgr('mylock', expires=30) as l: ... sleep(10) ... l.renew(expires=60) # Add 60 seconds more time to 'mylock' expiration ... l.main_lock.refresh_from_db() ... print(l.main_lock.expires_seconds) # Output: 79 ... l.renew('lockx', expires=60) # Add 60 seconds more time to 'lockx' expiration
- Parameters
lock (Lock) – Name of the lock to renew
lock – A
Lock
object to renewexpires (int) – (Default: 120) If not add_time, then this is the new expiration time in seconds from now. If add_time, then this many seconds will be added to the expiration time of the lock.
add_time (bool) – (Default:
True
) If True, thenexpires
seconds will be added to the existing lock expiration time, instead of setting the expiration time tonow + expires
Extra Keyword Arguments
- Key bool create
(Default:
True
) If True, then create a new lock if it doesn’t exist / already expired- Key str locked_by
(Default: system hostname) What server/app is trying to obtain this lock?
- Key int lock_process
(Optional) The process ID requesting the lock
Exceptions
- Raises
LockNotFound – Raised if the requested
lock
doesn’t exist / is already expired andcreate
is False.- Return Lock lock
The
Lock
object which was renewed
-
wait
= None¶ How long to wait for a lock before giving up. If this is
None
then waiting will be disabled
-
-
lockmgr.lockmgr.
get_lock
(name, expires: Optional[int] = 600, locked_by: str = None, lock_process: int = None) → lockmgr.models.Lock[source]¶ READ THIS: It’s best to use
LockMgr
as it automatically handles locking and unlocking usingwith
.Calls
clean_locks()
to remove any expired locks, checks for any existing locks using a FOR UPDATE transaction, then attempts to obtain a lock using the Lock modelpayments.models.Lock
If
name
is already locked, thenLocked
will be raised.Otherwise, if it was successfully locked, a
payments.models.Lock
object for the requested lock name will be returned.Usage:
>>> try: # Obtain a lock on 'mylock', with an automatic expiry of 60 seconds. ... mylock = get_lock('mylock', 60) ... print('Successfully locked mylock') ... except Locked as e: ... print('Failed to lock. Reason: ', type(e), str(e)) ... finally: # Regardless of whether there was an exception or not, remember to remove the lock! ... print('Removing lock on "mylock"') ... unlock(mylock)
- Parameters
name (str) – A unique name to identify your lock
expires (int) – (Default: 600 sec) How long before this lock is considered stale and forcefully released? Set this to
0
for a lock which will never expire (must manually callunlock()
)locked_by (str) – (Default: system hostname) What server/app is trying to obtain this lock?
lock_process (int) – (Optional) The process ID requesting the lock
- Raises
Locked – If the requested lock
name
is already locked elsewhere,Locked
will be raised- Return Lock lock
If successfully locked, will return the
payments.models.Lock
of the requested lock.
-
lockmgr.lockmgr.
is_locked
(name: Union[lockmgr.models.Lock, str]) → bool[source]¶ Cleans expired locks, then returns
True
if the given lock keyname
exists, otherwiseFalse
-
lockmgr.lockmgr.
renew_lock
(lock: Union[str, lockmgr.models.Lock], expires: int = 600, add_time: bool = False, **kwargs) → lockmgr.models.Lock[source]¶ Renew an existing lock for more expiry time.
Note: This function will NOT reduce a lock’s expiry time, only lengthen. If
add_time
isFalse
, and the new expiration timeexpires
is shorter than the lock’s existing expiration time, then the lock’s expiry time will be left untouched.Example - Renew an existing lock:
>>> lk = get_lock('my_app:somelock', expires=10) >>> sleep(5) >>> lk = renew_lock(lk, 20) # Change the expiry time to 20 seconds from now >>> sleep(15) >>> is_locked('my_app:somelock') # 15 seconds later, the lock is still locked True
Example - Try to renew, but get a new lock if it’s already been released:
>>> lk = get_lock('my_app:somelock', expires=5) >>> sleep(10) >>> lk = renew_lock(lk, 20, create=True) # If the lock is expired/non-existant, make a new lock >>> sleep(15) >>> is_locked('my_app:somelock') # 15 seconds later, the lock is still locked True
- Parameters
lock (Lock) – Name of the lock to renew
lock – A
Lock
object to renewexpires (int) – (Default: 600) If not add_time, then this is the new expiration time in seconds from now. If add_time, then this many seconds will be added to the expiration time of the lock.
add_time (bool) – (Default:
False
) If True, thenexpires
seconds will be added to the existing lock expiration time, instead of setting the expiration time tonow + expires
- Key bool create
(Default:
False
) If True, then create a new lock if it doesn’t exist / already expired.- Key str locked_by
(Default: system hostname) What server/app is trying to obtain this lock?
- Key int lock_process
(Optional) The process ID requesting the lock
- Raises
LockNotFound – Raised if the requested
lock
doesn’t exist / is already expired andcreate
is False.- Return Lock lock
The
Lock
object which was renewed
-
lockmgr.lockmgr.
set_lock
(*locks, timeout=600, fail=False, renew=True, create=True, **options) → lockmgr.lockmgr.LockSetResult[source]¶ This function is for advanced users, offering multiple lock creation, renewing, along with “all or nothing” locking with database rollback via the argument
fail
.Unlike other lock management functions, set_lock returns a
LockSetResult
object, which is designed to allow you to see clearly as to what locks were created, renewed, or skipped.Example Usage
Let’s set two locks,
hello
andworld
.>>> res = set_lock('hello', 'world') >>> res['locks'] [<Lock name='hello' locked_by='example.org' locked_until='2019-11-22 02:01:55.439390+00:00'>, <Lock name='world' locked_by='example.org' locked_until='2019-11-22 02:01:55.442734+00:00'>] >>> res['counts'] {'created': 2, 'renewed': 0, 'skip_create': 0, 'skip_renew': 0}
If we run
set_lock
again with the same arguments, we’ll still get the locks list, but we’ll see the counts show that they were renewed instead of created.>>> x = set_lock('hello', 'world') >>> x['locks'] [<Lock name='hello' locked_by='example.org' locked_until='2019-11-22 02:03:06.762620+00:00'>, <Lock name='world' locked_by='example.org' locked_until='2019-11-22 02:03:06.766804+00:00'>] >>> x['counts'] {'created': 0, 'renewed': 2, 'skip_create': 0, 'skip_renew': 0}
Since the result is an object, you can also access attributes via dot notation, as well as dict-like notation.
We can see inside of the
statuses
list - the action that was taken on each lock we specified, so we can see what locks were created, renewed, or skipped etc.>>> x.statuses[0] ('hello', {'was_locked': True, 'status': 'extend', 'locked': True}) >>> x.statuses[1] ('world', {'was_locked': True, 'status': 'extend', 'locked': True})
- Parameters
locks (str) – One or more lock names, as positional arguments, to create or renew.
timeout (int) – On existing locks, update locked_until to
now + timeout
(seconds)fail (bool) – (Default: False) If
True
, all lock creations will be rolled back if an existing lock is encountered, andLockFail
will be raised.renew (bool) – (Default: True) If
True
, any existing locks inlocks
will be renewed tonow + timeout
(seconds). If False, existing locks will just be skipped.create (bool) – (Default: True) If
True
, any names inlocks
which aren’t yet locked, will have a lock created for them, with their expiry set totimeout
seconds from now.
- Key str locked_by
(Default: system hostname) What server/app is trying to obtain this lock?
- Key int process_id
(Optional) The process ID requesting the lock
- Return LockSetResult results
A
LockSetResult
object containing the results of the set_lock operation.
-
lockmgr.lockmgr.
unlock
(lock: Union[lockmgr.models.Lock, str])[source]¶ Releases a given lock - either specified as a string name, or as a
payments.models.Lock
object.Usage:
>>> mylock = get_lock('mylock', expires=60) >>> unlock('mylock') # Delete the lock by name >>> unlock(mylock) # Or by Lock object.
API Docs (lockmgr.lockmgr)¶
Functions
Deletes expired
Lock
objects.
get_lock
(name[, expires, locked_by, …])READ THIS: It’s best to use
LockMgr
as it automatically handles locking and unlocking usingwith
.
is_locked
(name)Cleans expired locks, then returns
True
if the given lock keyname
exists, otherwiseFalse
renew_lock
(lock[, expires, add_time])Renew an existing lock for more expiry time.
set_lock
(*locks[, timeout, fail, renew, create])This function is for advanced users, offering multiple lock creation, renewing, along with “all or nothing” locking with database rollback via the argument
fail
.
unlock
(lock)Releases a given lock - either specified as a string name, or as a
payments.models.Lock
object.Classes
LockMgr
(name[, expires, locked_by, …])LockMgr is a wrapper class for the various locking functions in this module, e.g.
Exceptions
Raised when a requested lock doesn’t exist
Raised when a lock already exists with the given name