django-garnett
Django Garnett is a field level translation library that allows you to store strings in multiple languages for fields in Django - with minimal changes to your models and without having to rewrite your code.
Want a demo? https://django-garnett.herokuapp.com/
Made with ♥ by
Aristotle Metadata
In summary it allows you to do this:
models.py | You can do this! |
By changing your models from this...
|
|
Tested on: - Django 3.1+ - Postgres, SQLite, MariaDB - Python 3.7+
Tested with the following libraries: - Django Rest Framework - django-reversion & django-reversion-compare - django-filters
Pros:
* Battletested in production - Aristotle Metadata built, support and uses this library for 2 separate products, served to government and enterprise clients!
* Fetching all translations for a model requires a single query
* Translations are stored in a single database field with the model
* All translations act like a regular field, so Model.field_name = "some string"
and print(Model.field_name)
work as you would expect
* Includes a configurable middleware that can set the current language context based on users cookies, query string or HTTP headers
* Works nicely with Django Rest Framework - translatable fields can be set as strings or as json dictionaries
* Works nicely with Django F()
and Q()
objects within the ORM - and when it doesn't we have a language aware LangF()
replacement you can use.
Cons:
* You need to alter the models, so you can't make third-party libraries translatable.
* It doesn't work on queryset.values_list
or queryset.values
natively - but we have an easy Queryset Mixin to add language support for both below.
Why write a new Django field translator?
A few reasons: * Most existing django field translation libraries are static, and add separate database columns or extra tables per translation. * Other libraries may not be compatible with common django libraries, like django-rest-framework. * We had a huge codebase that we wanted to upgrade to be multilingual - so we needed a library that could be added in without requiring a rewriting every access to fields on models, and only required minor tweaks.
Note: Field language is different to the django display language. Django can be set up to translate your pages based on the users browser and serve them with a user interface in their preferred language.
Garnett does not use the browser language by design - a user with a French browser may want the user interface in French, but want to see content in English or French based on their needs.
How to install
- Add
django-garnett
to your dependencies. eg.pip install django-garnett
-
Convert your chosen field using the
Translated
function- For example:
title = fields.Translated(models.CharField(*args))
- For example:
-
Add
GARNETT_TRANSLATABLE_LANGUAGES
(a callable or list of language codes) to your django settings. > Note: At the moment there is no way to allow "a user to enter any language". - Add
GARNETT_DEFAULT_TRANSLATABLE_LANGUAGE
(a callable or single language code) to your settings. - Re-run
django makemigrations
, perform a data migration to make your existing data translatable (See 'Data migrations') &django migrate
for any apps you've updated. - Thats mostly it.
You can also add a few optional value adds:
-
(Optional) Add a garnett middleware to take care of field language handling:
-
You want to capture the garnett language in a context variable available in views use:
garnett.middleware.TranslationContextMiddleware
-
You want to capture the garnett language in a context variable available in views, and want to raise a 404 if the user requests an invalid language use:
garnett.middleware.TranslationContextNotFoundMiddleware
-
(Future addition) You want to capture the garnett language in a context variable available in views, and want to redirect to the default language if the user requests an invalid language use:
garnett.middleware.TranslationContextRedirectDefaultMiddleware
-
If you want to cache the current language in session storage use
garnett.middleware.TranslationCacheMiddleware
after one of the above middleware (this is useful with the session selector mentioned below)
-
-
(Optional) Add the
garnett
app to yourINSTALLED_APPS
to use garnett's template_tags. If this is installed beforedjango.contrib.admin
it also include a language switcher in the Django Admin Site. -
(Optional) Add a template processor:
- Install
garnett.context_processors.languages
this will addgarnett_languages
(a list of availableLanguage
s) andgarnett_current_language
(the currently selected language).
- Install
-
(Optional) Add a custom translation fallback:
By default, if a language isn't available for a field, Garnett will show a mesage like:
No translation of this field available in English
You can override this either by creating a custom fallback method:
Translated(CharField(max_length=150), fallback=my_fallback_method))
Wheremy_fallback_method
takes a dictionary of language codes and corresponding strings, and returns the necessary text.Additionally, you can customise how django outputs text in templates by creating a new
TranslationStr
class, and overriding the__html__
method.
Data migrations
If you have lots of existing data (and if you are using this library you probably do) you will need to perform data migrations to make sure all of your existing data is multi-lingual aware. Fortunately, we've added some well tested migration utilities that can take care of this for you.
Once you have run django-admin makemigrations
you just need to add the step_1_safe_encode_content
before and step_2_safe_prepare_translations
after your schema migrations, like in the following example:
# Generated by Django 3.1.13 on 2022-01-11 10:13
from django.db import migrations, models
import garnett.fields
import library_app.models
#### Add this line in
from garnett.migrate import step_1_safe_encode_content, step_2_safe_prepare_translations
#### Define the models and fields you want ot migrate
model_fields = {
"book": ["title", "description"],
}
class Migration(migrations.Migration):
dependencies = [
("library_app", "0001_initial"),
]
operations = [
## Add this operation at the start
step_1_safe_encode_content("library_app", model_fields),
## These are the automatically generated migrations
migrations.AlterField( # ... migrate title to TranslatedField),
migrations.AlterField( # ... migrate description to TranslatedField),
## Add this operation at the start
step_2_safe_prepare_translations("library_app", model_fields),
]
Language
vs language
Django Garnett uses the python langcodes
library to determine more information about the languages being used - including the full name and local name of the language being used. This is stored as a Language
object.
Django Settings options:
GARNETT_DEFAULT_TRANSLATABLE_LANGUAGE
- Stores the default language to be used for reading and writing fields if no language is set in a context manager or by a request.
- By default it is 'en-AU' the language code for 'Strayan, the native tongue of inhabitants of 'Straya (or more commonly known as Australia).
- This can also be a callable that returns list of language codes. Combined with storing user settings in something like (django-solo)[https://github.com/lazybird/django-solo] users can dynamically add or change their language settings.
- default:
'en-AU'
GARNETT_TRANSLATABLE_LANGUAGES
:- Stores a list of language codes that users can use to save against TranslatableFields.
- This can also be a callable that returns list of language codes. Combined with storing user settings in something like (django-solo)[https://github.com/lazybird/django-solo] users can dynamically add or change their language settings.
- default
[GARNETT_DEFAULT_TRANSLATABLE_LANGUAGE]
GARNETT_REQUEST_LANGUAGE_SELECTORS
:- A list of string modules that determines the order of options used to determine the language selected by the user. The first selector found is used for the language for the request, if none are found the DEFAULT_LANGUAGE is used. These can any of the following in any order:
garnett.selector.query
: Checks theGARNETT_QUERY_PARAMETER_NAME
for a language to displaygarnett.selector.cookie
: Checks for a cookie calledGARNETT_LANGUAGE_CODE
for a language to display. Note: you cannot change this cookie name.garnett.selector.session
: Checks for a session keyGARNETT_LANGUAGE_CODE
for a language to display. Note: you cannot change this key name.garnett.selector.header
: Checks for a HTTP Header calledX-Garnett-Language-Code
for a language to display. Note: you cannot change this Header name.garnett.selector.browser
: Uses Django'sget_language
function to get the users browser/UI language as determined by Django.
- For example, if you only want to check headers and cookies in that order, set this to
['garnett.selectors.header', 'garnett.selectors.cookie']
. - default:
['garnett.selectors.header', 'garnett.selectors.query', 'garnett.selectors.cookie']
- A list of string modules that determines the order of options used to determine the language selected by the user. The first selector found is used for the language for the request, if none are found the DEFAULT_LANGUAGE is used. These can any of the following in any order:
GARNETT_QUERY_PARAMETER_NAME
:- The query parameter used to determine the language requested by a user during a HTTP request.
- default:
glang
GARNETT_ALLOW_BLANK_FALLBACK_OVERRIDE
- If set to true, when allpying the current language middleware, this will check for an extra get URL parameter to override the fallback to return blank strings where a languge has no content. This is useful for APIs
- default: False
Advanced Settings (you probably don't need to adjust these)
* GARNETT_TRANSLATABLE_FIELDS_PROPERTY_NAME
:
* Garnett adds a property to all models that returns a list of all TranslatableFields. By default, this is 'translatable_fields', but you can customise it here if you want.
* default: translatable_fields
* GARNETT_TRANSLATIONS_PROPERTY_NAME
:
* Garnett adds a property to all models that returns a dictionary of all translations of all TranslatableFields. By default, this is 'translations', but you can customise it here if you want.
* default: translations
Using Garnett
If you did everything above correctly, garnett should for the most part "just work".
Switching the active language
Garnett comes with a handy context manager that can be used to specify the current language. In any place where you want to manually control the current language, wrap your code in set_field_language
and garnett will correctly store the language. This can be nested, or you can change the language for a context multiple times before saving.
from garnett.context import set_field_language
greeting = Greeting(text="Hello", target="World")
with set_field_language("en"):
greeting.text = "Hello"
with set_field_language("fr"):
greeting.text = "Bonjour"
greeting.save()
Using Garnett with values_list
or values
This is one of the areas that garnett doesn't work immediately, but there is a solution.
In the places you are using values lists or values
, wrap any translated field in an L-expression and the values list will return correctly. For example:
from garnett.expressions import L
Book.objects.values_list(L("title"))
Book.objects.values(L("title"))
Using Garnett with Django-Rest-Framework
As TranslationField
s are based on JSONField, by default Django-Rest-Framework renders these as a JSONField, which may not be ideal.
You can get around this by using the TranslatableSerializerMixin
as the first mixin, which adds the necessary hooks to your serializer. This will mean class changes, but you won't need to update or override every field.
For example:
from rest_framework import serializers
from library_app import models
from garnett.ext.drf import TranslatableSerializerMixin
class BookSerializer(TranslatableSerializerMixin, serializers.ModelSerializer):
class Meta:
model = models.Book
fields = "__all__"
This will allow you to set the value for a translatable as either a string for the active langauge, or by setting a dictionary that has all languages to be saved (note: this will override the existing language set). For example:
To override just the active language:
curl -X PATCH ... -d "{ \"title\": \"Hello\"}"
To specifically override a single language (for example, Klingon):
curl -X PATCH ... -H "X-Garnett-Language-Code: tlh" -d "{ \"title\": \"Hello\"}"
To override all languages:
curl -X PATCH ... -d "{ \"title\": {\"en\": \"Hello\", \"fr\": \"Bonjour\"}}"
Using Garnett with django-reversion and django-reversion-compare
There are a few minor tweaks required to get Garnett to operate properly with django-reversion and django-reversion-compare based on how they serialise and display data.
This is because Garnett does not use the same 'field.attname' and 'field.name' which means serialization in Django will not work correctly.
To get django-reversion to work you will need to use a translation-aware serialiser and apply a patch to ensure that django-reversion-compare can show the right information.
An example json translation-aware serializer is included with Garnett and this can be applied with the following two settings in settings.py
:
# In settings.py
GARNETT_PATCH_REVERSION_COMPARE = True
SERIALIZATION_MODULES = {"json": "garnett.serializers.json"}
TranslatedFields will list the history and changes in json, but it does do comparisons correctly.
Why call it Garnett?
- Libraries need a good name.
- Searching for "Famous Translators" will tell you about Constance Garnett.
- Searching for "Django Garnett" showed there was no python library with this name.
- It did however talk about Garnet Clark (also spelled Garnett), a jazz pianist who played with Django Reinhart - the namesake of the Django Web Framework.
- Voila - a nice name
Warnings
contains
acts likeicontains
on SQLite only when doing a contains query it does a case insensitive search. I don't know why - https://www.youtube.com/watch?v=PgGNWRtceag- Due to how django sets admin form fields you will not get the admin specific widgets like
AdminTextAreaWidget
on translated fields in the django admin site by default. They can however be specified explicitly on the corresponding admin model form.
Want to help maintain this library?
There is a /dev/
directory with a docker-compose stack you can ues to bring up a database and clean development environment.
Want other options?
There are a few good options for adding translatable strings to Django that may meet other use cases. We've included a few other options here, their strengths and why we didn't go with them.
Pros: this library lets you apply translations to external apps, without altering their models.
Cons: each translation adds an extra column, which means languages are specified in code, and can't be altered by users later. * django-translated-fields
Pros: uses a great context processor for switching lanugages (which is where we got the idea for ours).
Cons: Languages are specified in django-settings, and can't be altered by users later. * django-parler
Pros: Django admin site support.
Cons: Languages are stored in a separate table and can't be altered by users later. Translated fields are specified in model meta, away from the fields definition which makes complex lookups harder.