Advanced C: Master and Demo Data
Master data is usually part of the technical or business requirements for the module. In other words, such data is often necessary for the module to work properly. This data will always be installed when installing the module.
On top of technical data, business data can be defined, e.g. countries, currencies, units of measure, as well as complete country localization (legal reports, tax definitions, chart of account), and much more…
In additional to master data, which are requirements for a module to work properly, we also like having data for demonstration purposes:
Help the sales representatives make their demos quickly.
Have a set of working data for developers to test new features and see how these new features look with data they might not have added themselves.
Test that the data is loaded correctly, without raising an error.
Setup most of the features to be used quickly when creating a new database.
Demo data is automatically loaded when you start the server if you don’t explicitly say you don’t want it. This can be done in the database manager or with the command line.
Reference: the documentation related to this topic can be found in Module Manifests.
Data is declared either in CSV or in XML. Each file containing data must be added in the manifest for them to be loaded.
The keys to use in the manifest to add new data are
data for the master data and
demo for the demo data. Both values should be a list of strings representing the relative paths to the files declaring the data.
Usually, demo data is in a
demo folder, views and actions are in a
views folder, security related data is in a
security folder, and other data is in a
If your work tree looks like this:
Your manifest should look like this:
Reference: the documentation related to this topic can be found in CSV data files.
The easiest way to declare simple data is by using the CSV format. This is however limited in terms of features: use it for long lists of simple models, but prefer XML otherwise.
Reference: the documentation related to this topic can be found in Data Files.
When the data to create is more complex it can be useful, or even necessary, to do it in XML.
During the Core Training, we saw in the Chapter 13: Inheritance chapter we could inherit (extend) an existing view. This was a special case of data extension: any data can be extended in a module.
When you are adding new fields to an existing model in a new module, you might want to populate those fields on the records created in the modules you are depending on. This is done by giving the
xml_id of the record you want to extend. It won’t replace it, in this case we will set the
field_c to the given value for both records.
Related fields can be set using the
ref key. The value of that key is the
xml_id of the record you want to link. Remember the
xml_id is composed of the name of the module where the data is first declared, followed by a dot, followed by the
id of the record (just the
id works too if you are in the module declaring it).
The value to assign to a field is not always a simple string and you might need to compute it. It can also be used to optimize the insertion of related values, or because a constraint forces you to add the related values in batch. See :Add X2many fields.
Sometimes, you need to call the ORM to do a
search. This is not feasible with the CSV format.
In this code snippet, it is needed because the master data depends on the localization installed.
You might also need to execute python code when loading data.
Add X2many fields
Reference: the documentation related to this topic can be found in
If you need to add related data in a One2many or a Many2many field, you can do so by using the
Accessing the data
There are multiple ways to access the master/demo data.
In python code, you can use the
env.ref(self, xml_id, raise_if_not_found=True) method. It returns the recordset linked to the
xml_id you specify.
In XML, you can use the
ref key like this
It will call the ref method, and store the id of the record returned on the field
related_id of the record of type
tutorial.example with id
In CSV, the title of the column must be suffixed with
In SQL, it is more complicated, see the advanced section.
What is the XML id?
Because we don’t want a column
xml_id in every single SQL table of the database, we need a mechanism to store it. This is done with the
It contains the name of the record (the
xml_id) along with the module in which it is defined, the model defining it, and the id of it.
The records created with the
noupdate flag won’t be updated when upgrading the module that created them, but it will be created if it didn’t exist yet.
Import as SQL
In some cases, it makes sense to do the import directly in SQL. This is however discouraged as it bypasses all the features of the ORM, computed fields (including metadata) and python constraints.