Patching datetime.date.today() across Python implementations

by Raphaël Barrois
in Python

tagged Python

Altering builtin functions and classes of Python during tests if often frowned upon, but can be actually required while writing unit tests.

The typical builtin requiring such alterations are the datetime.date.today and datetime.datetime.now() functions, which return (obviously) time-dependent values.

We'll take a look at this issue throught the (now standard) mock library.

The problem

The naive approach, mock.patch('datetime.date.today', return_value=MY_NOW), fails:

>>> target = datetime.datetime(2009, 1, 1)
>>> with mock.patch('datetime.datetime.now', return_value=target):
...   print(datetime.datetime.now())
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.2/site-packages/mock.py", line 1359, in __enter__
    setattr(self.target, self.attribute, new_attr)
TypeError: can't set attributes of built-in/extension type 'datetime.datetime'

This error indicates that datetime.date is actually implemented in C, not in Python; it is thus impossible to alter it through setattr

A simple solution

Based on a post by Ned Batchelder, let's try replacing the datetime.datetime class altogether:

>>> target = datetime.datetime(2009, 1, 1)
>>> with mock.patch.object(datetime, 'datetime', mock.Mock(wraps=datetime.datetime)) as patched:
...     patched.now.return_value = target
...     print(datetime.datetime.now())
...
2009-01-01 00:00:00
>>> print(datetime.datetime.now())
2013-04-21 20:37:26

What happens here?

We replace the datetime.datetime class with a mock that simply wraps the class, then alter the return value of the now() class method.

More problems

This solution looks simple, efficient; yet it has another problem: it breaks calls to isinstance(x, datetime.datetime):

>>> target = datetime.datetime(2009, 1, 1)
>>> with mock.patch.object(datetime, 'datetime', mock.Mock(wraps=datetime.datetime)) as patched:
...     isinstance(datetime.datetime.now(), datetime.datetime)
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: isinstance() arg 2 must be a type or typle of types

Note

Such isinstance checks are performed in PyPy when adding/substracting datetimes.

The problem comes from having replaced the actual datetime.datetime class with an object (a mock.Mock instance).

A better solution

It appears that the mock library can only produce objects, not classes. We'll have to create our own datetime.datetime subclass:

# mock_dt.py
import datetime
import mock

real_datetime_class = datetime.datetime

def mock_datetime_now(target, dt):
    class DatetimeSubclassMeta(type):
        @classmethod
        def __instancecheck__(mcs, obj):
            return isinstance(obj, real_datetime_class)

    class BaseMockedDatetime(real_datetime_class):
        @classmethod
        def now(cls, tz=None):
            return target.replace(tzinfo=tz)

        @classmethod
        def utcnow(cls):
            return target

    # Python2 & Python3 compatible metaclass
    MockedDatetime = DatetimeSubclassMeta('datetime', (BaseMockedDatetime,), {})

    return mock.patch.object(dt, 'datetime', MockedDatetime)

The process is somewhat complicated, since we're replacing an existing class with one of its subclasses, yet want isinstance checks to pass. This goes through customization of the __instancecheck__ method, that must be defined in a metaclass.

We now have a fully working mock:

>>> target = datetime.datetime(2009, 1, 1)
>>> with mock_datetime_now(target, datetime):
...     print(datetime.datetime.now())
...     print(isinstance(datetime.datetime.now(), datetime.datetime))
...
2009-01-01 00:00:00
True
>>> print(datetime.datetime.now())
2013-04-21 20:37:26

The fully working and documented code can be found here: https://gist.github.com/rbarrois/5430921

Alternate approach

Since finding all calls to today / now in a project's codebase can be quite cumbersome, and they are used massively around, a simpler approach is to write custom functions in a project-side module.

This has been performed by Django through its django.utils.timezone.now() function, that returns datetime.datetime.now localized to the current timezone.

Note

Projects should always use timezone-aware datetimes, defaulting to UTC. This project-wide module would be the perfect place for that.

Once this global module is setup, mocking it becomes much simpler:

jan1 = datetime.datetime(2008, 1, 1, tzinfo=pytz.UTC)

with mock.patch('my_project.time.now', return_value=jan1):
    do_something()