Miscellaneous tools for django.

Look also at the siblings project: django-cms-tools (Tools/helpers around Django-CMS).

| | https://pypi.python.org/pypi/django-tools/ | | | github.com/jedie/django-tools/actions | | | codecov.io/gh/jedie/django-tools | | | coveralls.io/r/jedie/django-tools |

(Logo contributed by @reallinfo see #16)

try-out

e.g.:

~$ git clone https://github.com/jedie/django-tools.git
~$ cd django-tools/
~/django-tools$ ./manage.py

existing stuff

Serve User Media File

Serve settings.MEDIA_ROOT files only for allowed users.

See separate README here: django_tools/serve_media_app

Mode Version Protect

Protect a model against overwriting a newer entry with an older one, by adding a auto increment version number.

See separate README here: django_tools/model_version_protect

OverwriteFileSystemStorage

A django filesystem storage that will overwrite existing files and can create backups, if content changed. usage:

class ExampleModel(models.Model):
    foo_file = models.FileField(
        storage=OverwriteFileSystemStorage(create_backups=True)
    )
    bar_image = models.ImageField(
        storage=OverwriteFileSystemStorage(create_backups=False)
    )

Backup made by appending a suffix and sequential number, e.g.:

• source....: foo.bar
• backup 1..: foo.bar.bak
• backup 2..: foo.bar.bak0
• backup 3..: foo.bar.bak1

Backup files are only made if file content changed. But at least one time!

Django Logging utils

Put this into your settings, e.g.:

from django_tools.unittest_utils.logging_utils import CutPathnameLogRecordFactory, FilterAndLogWarnings

# Filter warnings and pipe them to logging system
# Warnings of external packages are displayed only once and only the file path.
warnings.showwarning = FilterAndLogWarnings()

# Adds 'cut_path' attribute on log record. So '%(cut_path)s' can be used in log formatter.
logging.setLogRecordFactory(CutPathnameLogRecordFactory(max_length=50))

LOGGING = {
    # ...
    'formatters': {
        'verbose': {
            'format': '%(levelname)8s %(cut_path)s:%(lineno)-3s %(message)s'
        },
    },
    # ...
}

(Activate warnings by, e.g.: export PYTHONWARNINGS=all)

ThrottledAdminEmailHandler

ThrottledAdminEmailHandler works similar as the origin django.utils.log.AdminEmailHandler but is will throttle the number of mails that can be send in a time range. usage e.g.:

LOGGING = {
    # ...
    "handlers": {
        "mail_admins": {
            "level": "ERROR",
            "class": "django_tools.log_utils.throttle_admin_email_handler.ThrottledAdminEmailHandler",
            "formatter": "email",
            "min_delay_sec": 20, # << -- skip mails in this time range
        },
        # ...
    },
    # ...
}

django_tools.template.loader.DebugCacheLoader

Insert template name as html comments, e.g.:

<!-- START 'foo/bar.html' -->
...
<!-- END 'foo/bar.html' -->

To use this, you must add django_tools.template.loader.DebugCacheLoader as template loader.

e.g.: Activate it only in DEBUG mode:

if DEBUG:
    TEMPLATES[0]["OPTIONS"]["loaders"] = [
        (
            "django_tools.template.loader.DebugCacheLoader", (
                'django.template.loaders.filesystem.Loader',
                'django.template.loaders.app_directories.Loader',
            )
        )
    ]

send text+html mails

A helper class to send text+html mails used the django template library.

You need two template files, e.g.:

• mail_test.txt
• mail_test.html

You have to specify the template file like this: template_base="mail_test.{ext}"

Send via Celery task:

# settings.py
SEND_MAIL_CELERY_TASK_NAME="mail:send_task"

from django_tools.mail.send_mail import SendMailCelery
SendMailCelery(
    template_base="mail_test.{ext}",
    mail_context={"foo": "first", "bar": "second"},
    subject="Only a test",
    recipient_list="[email protected]"
).send()

Send without Celery:

from django_tools.mail.send_mail import SendMail
SendMail(
    template_base="mail_test.{ext}",
    mail_context={"foo": "first", "bar": "second"},
    subject="Only a test",
    recipient_list="[email protected]"
).send()

See also the existing unittests:

• django_tools_tests/test_email.py

Delay tools

Sometimes you want to simulate when processing takes a little longer. There exists django_tools.debug.delay.SessionDelay and django_tools.debug.delay.CacheDelay for this. The usage will create logging entries and user messages, if user is authenticated.

More info in seperate django_tools/debug/README.creole file.

Filemanager library

Library for building django application like filemanager, gallery etc.

more info, read ./filemanager/README.creole

per-site cache middleware

Similar to django UpdateCacheMiddleware and FetchFromCacheMiddleware, but has some enhancements: 'per site cache' in ./cache/README.creole

smooth cache backends

Same as django cache backends, but adds cache.smooth_update() to clears the cache smoothly depend on the current system load. more info in: 'smooth cache backends' in ./cache/README.creole

local sync cache

Keep a local dict in a multi-threaded environment up-to-date. Usefull for cache dicts. More info, read DocString in ./local_sync_cache/local_sync_cache.py.

threadlocals middleware

For getting request object anywhere, use ./middlewares/ThreadLocal.py

Dynamic SITE_ID middleware

Note: Currently not maintained! TODO: Fix unittests for all python/django version

Set settings.SITE_ID dynamically with a middleware base on the current request domain name. Domain name alias can be specify as a simple string or as a regular expression.

more info, read ./dynamic_site/README.creole.

StackInfoStorage

Message storage like LegacyFallbackStorage, except, every message would have a stack info, witch is helpful, for debugging. Stack info would only be added, if settings DEBUG or MESSAGE_DEBUG is on. To use it, put this into your settings:

MESSAGE_STORAGE = "django_tools.utils.messages.StackInfoStorage"

More info, read DocString in ./utils/messages.py.

limit to usergroups

Limit something with only one field, by selecting:

• anonymous users
• staff users
• superusers
• ..all existing user groups..

More info, read DocString in ./limit_to_usergroups.py

permission helpers

See django_tools.permissions and unittests: django_tools_tests.test_permissions

form/model fields

• Directory field - check if exist and if in a defined base path
• language code field with validator
• Media Path field browse existign path to select and validate input
• sign seperated form/model field e.g. comma seperated field
• static path field
• url field A flexible version of the original django form URLField

unittests helpers

Selenium Test Cases

There are Firefox and Chromium test cases, with and without django StaticLiveServerTestCase!

Chromium + StaticLiveServer example:

from django_tools.selenium.chromedriver import chromium_available
from django_tools.selenium.django import SeleniumChromiumStaticLiveServerTestCase

@unittest.skipUnless(chromium_available(), "Skip because Chromium is not available!")
class ExampleChromiumTests(SeleniumChromiumStaticLiveServerTestCase):
    def test_admin_login_page(self):
        self.driver.get(self.live_server_url + "/admin/login/")
        self.assert_equal_page_title("Log in | Django site admin")
        self.assert_in_page_source('<form action="/admin/login/" method="post" id="login-form">')
        self.assert_no_javascript_alert()

Firefox + StaticLiveServer example:

from django_tools.selenium.django import SeleniumFirefoxStaticLiveServerTestCase
from django_tools.selenium.geckodriver import firefox_available

@unittest.skipUnless(firefox_available(), "Skip because Firefox is not available!")
class ExampleFirefoxTests(SeleniumFirefoxStaticLiveServerTestCase):
    def test_admin_login_page(self):
        self.driver.get(self.live_server_url + "/admin/login/")
        self.assert_equal_page_title("Log in | Django site admin")
        self.assert_in_page_source('<form action="/admin/login/" method="post" id="login-form">')
        self.assert_no_javascript_alert()

Test cases without StaticLiveServer:

from django_tools.selenium.chromedriver import SeleniumChromiumTestCase
from django_tools.selenium.geckodriver import SeleniumFirefoxTestCase

See also existing unitests here:

• /django_tools/django_tools_tests/test_unittest_selenium.py

Setup Web Drivers

Selenium test cases needs the browser and the web driver.

SeleniumChromiumTestCase and SeleniumFirefoxTestCase will automaticly install the web driver via webdriver-manager

There is a small CLI (called django_tools_selenium) to check / install the web drivers, e.g.:

~/django-tools$ poetry run django_tools_selenium install
~/django-tools$ poetry run django_tools_selenium info

Mockup utils

Create dummy PIL/django-filer images with Text, see:

• /django_tools/unittest_utils/mockup.py

usage/tests:

• /django_tools_tests/test_mockup.py

Model instance unittest code generator

Generate unittest code skeletons from existing model instance. You can use this feature as django manage command or as admin action.

Usage as management command, e.g.:

$ ./manage.py generate_model_test_code auth.
...
#
# pk:1 from auth.User <class 'django.contrib.auth.models.User'>
#
user = User.objects.create(
    password='pbkdf2_sha256$36000$ybRfVQDOPQ9F$jwmgc5UsqRQSXxJs/NrZeTLguieUSSZfaSZbMmC+L5w=', # CharField, String (up to 128)
    last_login=datetime.datetime(2018, 4, 24, 8, 27, 49, 578107, tzinfo=<UTC>), # DateTimeField, Date (with time)
    is_superuser=True, # BooleanField, Boolean (Either True or False)
    username='test', # CharField, String (up to 150)
    first_name='', # CharField, String (up to 30)
    last_name='', # CharField, String (up to 30)
    email='', # CharField, Email address
    is_staff=True, # BooleanField, Boolean (Either True or False)
    is_active=True, # BooleanField, Boolean (Either True or False)
    date_joined=datetime.datetime(2018, 3, 6, 17, 15, 50, 93136, tzinfo=<UTC>), # DateTimeField, Date (with time)
)
...

create users

/unittest_utils/user.py:

• django_tools.unittest_utils.user.create_user() - create users, get_super_user
• django_tools.unittest_utils.user.get_super_user() - get the first existing superuser

Isolated Filesystem decorator / context manager

django_tools.unittest_utils.isolated_filesystem.isolated_filesystem acts as either a decorator or a context manager. Useful to for tests that will create files/directories in current work dir, it does this:

• create a new temp directory
• change the current working directory to the temp directory
• after exit:
• Delete an entire temp directory tree

usage e.g.:

from django_tools.unittest_utils.isolated_filesystem import isolated_filesystem

with isolated_filesystem(prefix="temp_dir_prefix"):
    open("foo.txt", "w").write("bar")

BaseUnittestCase

django_tools.unittest_utils.unittest_base.BaseUnittestCase contains some low-level assert methods:

• assertEqual_dedent()

Note: assert methods will be migrated to: django_tools.unittest_utils.assertments in the future!

django_tools.unittest_utils.tempdir contains TempDir, a Context Manager Class:

with TempDir(prefix="foo_") as tempfolder:
    # create a file:
    open(os.path.join(tempfolder, "bar"), "w").close()

# the created temp folder was deleted with shutil.rmtree()

usage/tests:

• /django_tools_tests/test_unittest_utils.py

DjangoCommandMixin

Helper to run shell commands. e.g.: "./manage.py cms check" in unittests.

usage/tests:

• /django_tools_tests/test_unittest_django_command.py

DOM compare in unittests

The Problem: You can’t easy check if e.g. some form input fields are in the response, because the form rendering use a dict for storing all html attributes. So, the ordering of form field attributes are not sorted and varied.

The Solution: You need to parse the response content into a DOM tree and compare nodes.

We add the great work of Gregor Müllegger at his GSoC 2011 form-rendering branch. You will have the following assert methods inherit from: django_tools.unittest_utils.unittest_base.BaseTestCase

• self.assertHTMLEqual() – for compare two HTML DOM trees
• self.assertDOM() – for check if nodes in response or not.
• self.assertContains() – Check if ond node occurs 'count’ times in response

More info and examples in ./django_tools_tests/test_dom_asserts.py

@set_string_if_invalid() decorator

Helper to check if there are missing template tags by set temporary 'string_if_invalid', see: https://docs.djangoproject.com/en/1.8/ref/templates/api/#invalid-template-variables

Usage, e.g.:

from django.test import SimpleTestCase
from django_tools.unittest_utils.template import TEMPLATE_INVALID_PREFIX, set_string_if_invalid

@set_string_if_invalid()
class TestMyTemplate(SimpleTestCase):
    def test_valid_tag(self):
        response = self.client.get('/foo/bar/')
        self.assertNotIn(TEMPLATE_INVALID_PREFIX, response.content)

You can also decorate the test method ;)

unittest_utils/signals.py

• SignalsContextManager connect/disconnet signal callbacks via with statement

unittest_utils/assertments.py

The file contains some common assert functions:

• assert_startswith - Check if test starts with prefix.
• assert_endswith - Check if text ends with suffix.
• assert_locmem_mail_backend - Check if current email backend is the In-memory backend.
• {{{assert_language_code() - Check if given language_code is in settings.LANGUAGES
• assert_installed_apps() - Check entries in settings.INSTALLED_APPS
• assert_is_dir - Check if given path is a directory
• assert_is_file - Check if given path is a file
• assert_path_not_exists - Check if given path doesn't exists

Speedup tests

Speedup test run start by disable migrations, e.g.:

from django_tools.unittest_utils.disable_migrations import DisableMigrations
MIGRATION_MODULES = DisableMigrations()

small tools

debug_csrf_failure()

Display the normal debug page and not the minimal csrf debug page. More info in DocString here: django_tools/views/csrf.py

import lib helper

additional helper to the existing importlib more info in the sourcecode: ./utils/importlib.py

http utils

Pimped HttpRequest to get some more information about a request. More info in DocString here: django_tools/utils/http.py

@display_admin_error

Developer helper to display silent errors in ModelAdmin.list_display callables. See: display_admin_error in decorators.py

upgrade virtualenv

A simple commandline script that calls pip install —-upgrade XY for every package thats installed in a virtualenv. Simply copy/symlink it into the root directory of your virtualenv and start it.

Note:Seems that this solution can't observe editables right.

To use it, without installing django-tools:

~/$ cd goto/your_env
.../your_env/$ wget https://github.com/jedie/django-tools/raw/master/django_tools/upgrade_virtualenv.py
.../your_env/$ chmod +x upgrade_virtualenv.py
.../your_env/$ ./upgrade_virtualenv.py

This script will be obsolete, if pip has a own upgrade command.

django_tools.utils.url.GetDict

Similar to origin django.http.QueryDict but:

• urlencode() doesn't add "=" to empty values: "?empty" instead of "?empty="
• always mutable
• output will be sorted (easier for tests ;)

More info, see tests: django_tools_tests/test_utils_url.py

SignedCookieStorage

Store information in signed Cookies, use django.core.signing. So the cookie data can't be manipulated from the client. Sources/examples:

• /django_tools/utils/client_storage.py
• /django_tools_tests/test_signed_cookie.py

Print SQL Queries

Print the used SQL queries via context manager.

usage e.g.:

from django_tools.unittest_utils.print_sql import PrintQueries

# e.g. use in unittests:
class MyTests(TestCase):
    def test_foobar(self):
        with PrintQueries("Create object"):
            FooBar.objects.create("name"=foo)

# e.g. use in views:
def my_view(request):
    with PrintQueries("Create object"):
        FooBar.objects.create("name"=foo)

the output is like:

_______________________________________________________________________________
 *** Create object ***
1 - INSERT INTO "foobar" ("name")
    VALUES (foo)
-------------------------------------------------------------------------------

SetRequestDebugMiddleware

middleware to add debug bool attribute to request object. More info: ./debug/README.creole

TracebackLogMiddleware

Put traceback in log by call logging.exception() on process_exception() Activate with:

MIDDLEWARE_CLASSES = (
    ...
    'django_tools.middlewares.TracebackLogMiddleware.TracebackLogMiddleware',
    ...
)

FnMatchIps() - Unix shell-style wildcards in INTERNAL_IPS / ALLOWED_HOSTS

settings.py e.g.:

from django_tools.settings_utils import FnMatchIps

INTERNAL_IPS = FnMatchIps(["127.0.0.1", "::1", "192.168.*.*", "10.0.*.*"])
ALLOWED_HOSTS = FnMatchIps(["127.0.0.1", "::1", "192.168.*.*", "10.0.*.*"])

StdoutStderrBuffer()

redirect stdout + stderr to a string buffer. e.g.:

from django_tools.unittest_utils.stdout_redirect import StdoutStderrBuffer

with StdoutStderrBuffer() as buffer:
    print("foo")
output = buffer.get_output() # contains "foo\n"

Management commands

permission_info

List all permissions for one django user. (Needs 'django_tools' in INSTALLED_APPS)

e.g.:

$ ./manage.py permission_info
No username given!

All existing users are:
foo, bar, john, doe

$ ./manage.py permission_info foo
All permissions for user 'test_editor':
    is_active    : yes
    is_staff     : yes
    is_superuser : no
[*] admin.add_logentry
[*] admin.change_logentry
[*] admin.delete_logentry
[ ] auth.add_group
[ ] auth.add_permission
[ ] auth.add_user
...

logging_info

Shows a list of all loggers and marks which ones are configured in settings.LOGGING:

$ ./manage.py logging_info

nice_diffsettings

Similar to django 'diffsettings', but used pretty-printed representation:

$ ./manage.py nice_diffsettings

database_info

Just display some information about the used database and connections:

$ ./manage.py database_info

list_models

Just list all existing models in app_label.ModelName format. Useful to use this in 'dumpdata' etc:

$ ./manage.py list_models

..all others…

There exist many miscellaneous stuff. Look in the source, luke!

Backwards-incompatible changes

Old changes archived in git history here:

• >=v0.47
• >=v0.35

v0.51

All Selenium helper are deprecated, please migrate to Playwright ;)

v0.50

Removed old selenium helper function, deprecated since v0.43

Make all Selenium web driver instances persistent for the complete test run session. This speedup tests and fixed some bugs in Selenium.

This result in the same browser/webdriver settings for all test classes!

v0.55

Move supported Django/Python min. versions to: * Django 4.1, 4.2, 5.1 * Python 3.11, 3.12

Django compatibility

(See also combinations for tox in pyproject.toml)

history

• v0.56.2
• 2024-08-25 - Bugfix: Remove empty package that shadows existing codes
• v0.56.1
• 2024-08-25 - Use typeguard in tests
• 2024-08-25 - Use cli_base update-readme-history
• 2024-08-25 - Update via manageprojects
• v0.56.0
• 2024-08-25 - Bugfix local test run with a real terminal ;)
• 2024-08-25 - Fix CI
• 2023-04-10 - Upgrade: use managed-django-projec, Remove deprecations, update supported versions
• v0.54.0
• 2022-09-15 - Bugfix version check
• 2022-08-23 - Replace README.creole with README.md
• 2022-08-26 - Run safety check in CI
• 2022-08-25 - NEW: SyslogHandler for easy logging to syslog

links

| Homepage | https://github.com/jedie/django-tools | | PyPi | https://pypi.python.org/pypi/django-tools/ |

donation

• paypal.me/JensDiemer
• Flattr This!
• Send Bitcoins to 1823RZ5Md1Q2X5aSXRC5LRPcYdveCiVX6F