Patching datetime.date.today() across Python implementations

On Sun 21 April 2013 | By Raphaël Barrois | Category : Python | Tags : Python

Altering builtin functions and classes of Python during tests if often frowned upon, but can be actually required while writing unit tests.

The typical builtin requiring such alterations are the datetime.date.today and datetime.datetime.now() functions, which return (obviously) time-dependent values.

We'll take a look at this issue throught the (now standard) mock library.

The problem

The naive approach, mock.patch('datetime.date.today', return_value=MY_NOW), fails:

>>> target = datetime.datetime(2009, 1, 1)
>>> with mock.patch('datetime.datetime.now', return_value=target):
...   print(datetime.datetime.now())
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.2/site-packages/mock.py", line 1359, in __enter__
    setattr(self.target, self.attribute, new_attr)
TypeError: can't set attributes of built-in/extension type 'datetime.datetime'

This error indicates that datetime.date is actually implemented in C, not in Python; it is thus impossible to alter it through setattr

A simple solution

Based on a post by Ned Batchelder, let's try replacing the datetime.datetime class altogether:

>>> target = datetime.datetime(2009, 1, 1)
>>> with mock.patch.object(datetime, 'datetime', mock.Mock(wraps=datetime.datetime)) as patched:
...     patched.now.return_value = target
...     print(datetime.datetime.now())
...
2009-01-01 00:00:00
>>> print(datetime.datetime.now())
2013-04-21 20:37:26

What happens here?

We replace the datetime.datetime class with a mock that simply wraps the class, then alter the return value of the now() class method.

More problems

This solution looks simple, efficient; yet it has another problem: it breaks calls to isinstance(x, datetime.datetime):

>>> target = datetime.datetime(2009, 1, 1)
>>> with mock.patch.object(datetime, 'datetime', mock.Mock(wraps=datetime.datetime)) as patched:
...     isinstance(datetime.datetime.now(), datetime.datetime)
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: isinstance() arg 2 must be a type or typle of types

Note

Such isinstance checks are performed in PyPy when adding/substracting datetimes.

The problem comes from having replaced the actual datetime.datetime class with an object (a mock.Mock instance).

A better solution

It appears that the mock library can only produce objects, not classes. We'll have to create our own datetime.datetime subclass:

# mock_dt.py
import datetime
import mock

real_datetime_class = datetime.datetime

def mock_datetime_now(target, dt):
    class DatetimeSubclassMeta(type):
        @classmethod
        def __instancecheck__(mcs, obj):
            return isinstance(obj, real_datetime_class)

    class BaseMockedDatetime(real_datetime_class):
        @classmethod
        def now(cls, tz=None):
            return target.replace(tzinfo=tz)

        @classmethod
        def utcnow(cls):
            return target

    # Python2 & Python3 compatible metaclass
    MockedDatetime = DatetimeSubclassMeta('datetime', (BaseMockedDatetime,), {})

    return mock.patch.object(dt, 'datetime', MockedDatetime)

The process is somewhat complicated, since we're replacing an existing class with one of its subclasses, yet want isinstance checks to pass. This goes through customization of the __instancecheck__ method, that must be defined in a metaclass.

We now have a fully working mock:

>>> target = datetime.datetime(2009, 1, 1)
>>> with mock_datetime_now(target, datetime):
...     print(datetime.datetime.now())
...     print(isinstance(datetime.datetime.now(), datetime.datetime))
...
2009-01-01 00:00:00
True
>>> print(datetime.datetime.now())
2013-04-21 20:37:26

The fully working and documented code can be found here: https://gist.github.com/rbarrois/5430921

Alternate approach

Since finding all calls to today / now in a project's codebase can be quite cumbersome, and they are used massively around, a simpler approach is to write custom functions in a project-side module.

This has been performed by Django through its django.utils.timezone.now() function, that returns datetime.datetime.now localized to the current timezone.

Note

Projects should always use timezone-aware datetimes, defaulting to UTC. This project-wide module would be the perfect place for that.

Once this global module is setup, mocking it becomes much simpler:

jan1 = datetime.datetime(2008, 1, 1, tzinfo=pytz.UTC)

with mock.patch('my_project.time.now', return_value=jan1):
    do_something()

Other articles

Setting up a private, team-wide PyPI repository

On Wed 29 August 2012 | By Raphaël Barrois | Category : Python | Tags : System ~ Python

When developing Python applications, it may be useful to store some applications in a private repository.

This provides several benefits:

  • Dependencies available even if PyPI and its mirrors are down
  • Storing custom forks of upstream packages
  • Providing private packages in a standardized manner.

For this purpose, three components are required ...

More…

Getting Firefox/Chrome to use system-wide certificates

On Mon 13 August 2012 | By Raphaël Barrois | Category : Linux | Tags : System ~ Web ~ Linux ~ Gentoo

While browsing the web, I was getting quite annoyed that firefox, chrome and console tools (wget, ...) each maintained a separate list of trusted certificates.

Which led me to dive into the arcane world of NSS, the "Network Security Services".

Note

This is a rather long post, which will walk you ...

More…

Xel.Mind — Random thoughts

On Fri 10 August 2012 | By Raphaël Barrois | Category : Misc | Tags : Me

After a couple of years spent thinking about "Well, I should write out stuff somewhere, for my future self", I've finally crossed the line.

What can you expect on this "Weblog"?

  • Random thoughts on technology
  • Articles around Python, Gentoo, Django, SysAdmin stuff, …
  • Every once in a while, considerations on ...

More…