How not to make your config file a pain in the ass – MicroPython edition

Externalizing the configuration is a good idea. This makes the reusability of your program much higher and a user doesn’t have (and shouldn’t have to) change values in the code.
This separation tends to show where your code is perhaps too tightly coupled and constants are referenced directly as they are directly available in the script scope. This makes the code behavior difficult to predict as the constants can be changed at any time by any code part and you may not see when or why this happens instantly. This complicates independent testing as the functions and classes may depend on the availability of he constant in the rather than being a parameter.
Putting the config values out of the way also shrinks the file – which is always a good thing for a better overview.

Goals for the configuration file:

• Easy as possible for a user that does not know how to write code or what syntax XML or JSON use.
• Easy integration into the code with known field names for auto completion and avoiding copy and paste the field names.
• Without a config file reader with static field and config file names as that would just double the maintenance.

I ended up using a simple config.py file with constants and comments for documentation. Not very fancy, but did fulfill all my requirements. It even allows to put derived constants there, too, as the setup values can be accessed when defining them. As the syntax is kept very basic this should make it very accessible for most users. I am pretty happy with it.

# =========================================================================
# CONFIGURATION - adapt values to your setup
LDR_PIN = 0  # (single) ADC pin of D1 Mini
LED_PIN = 4  # change to your pin
LED_COUNT = 8  #  amount of LEDs connected at given pin
COLOR = (255, 120, 80)  #  RGB color values - upscale until highest is 255
MAX_BRIGHT = 50  #  maximum brightness of LEDs

# ==========================================================================
# DERIVED VALUES - DO NOT CHANGE (until you know what you are doing)
RGB = tuple(0 if v not in range(1, 256) else round(v / 100 * MAX_BRIGHT) for v in COLOR)
COLOR_STEPS = max(RGB)

Another solution for better encapsulation of the configuration in the program code is to use class inheritance. The user setup is done as fields in the class BaseConfig. For better overview this is also the only class in the file. In another file the class DefaultConfig which extends BaseConfig is defined. In it the derived setup values and other constants are defined.
The program itself uses an instance of DefaultConfig. This makes sure all values are included in the config and the user doesn’t even see the program code or the calculation of the derived setup values. Additionally the instance of the config is only available in the central program class and there are no global config values in the script. If another class needs a configuration value it gets it as parameter.
The following code blocks illustrate this setup.

# =========================================================================
# CONFIGURATION - adapt values to your setup
class BaseConfig:
    LDR_PIN = 0  # (single) ADC pin of D1 Mini
    LED_PIN = 4  # change to your pin
    LED_COUNT = 8  #  amount of LEDs connected at given pin
    COLOR = (255, 120, 80)  #  RGB color values - upscale until highest is 255
    MAX_BRIGHT = 50  #  maximum brightness of LEDs

from config import BaseConfig

class DefaultConfig:
    # ==========================================================================
    # DERIVED VALUES - DO NOT CHANGE (until you know what you are doing)
    RGB = tuple(0
                if v not in range(1, 256)
                else round(v / 100 * BaseConfig.MAX_BRIGHT) 
                for v in BaseConfig.COLOR)
    COLOR_STEPS = max(RGB)

from machine import ADC, Pin
from neopixel import NeoPixel
from config_generator import DefaultConfig


class NightLight:
    def __init__(self):
        self.cfg = DefaultConfig()
        self.light_sensor = ADC(self.cfg.LIGHT_SENSOR_PIN)
        self.pixels = NeoPixel(Pin(self.cfg.LED_PIN, self.cfg.LED_COUNT))
        # other class in same file needs config values - passed as params
        self.steps = Steps(self.cfg.RGB, self.cfg.COLOR_STEPS)

    def run(self):
       # do stuff


class Steps:
    def __init__(self, rgb: (int, int, int), steps: int):
        self.step_amount = steps
        self.rgb = rgb

    def calculate_stuff(self):
        # calculates stuff

Discarded possibilities for this case:

Dataclasses, YAML support are not natively available in MicroPython (get it here).

I don’t see the benefit using pickle for my project. The serialization and deserialization of the pickle file is unnecessary for my task and just more overhead. The raw config values have to be stored at another location additionally of course. A config value check could be added and only valid configuration values could be saved, but that is out of scope for this now.
Additionally using pickle is unsafe, as unknown pickles should not be unpickled as they can taste bad 😛 – or even execute code you dont know about.

The syntax of XML is not very inviting for the intended simple use. There is a MicroPython module xmltoc which supports basic XML handling, but this doesn’t seem to be in an official release yet.

JSON files are supported by MicroPython and it is pretty easy to handle them. I use it a lot for the MQTT protocol in Python and JavaScript on Node-Red and gave it a try for this, too. When loading a .json file you end up with a dict, but you still have to access it with a string for the key, which means a lot of copy and paste and error prone code.
I also checked collections.namedtuple, where you can – as you would expect – give the fields of the underlying tuple a name. But sadly I could not find a way to make the fields known to the IDE and use auto completion. So again copy and paste of the field names would have been necessary. Named tuples help with the readility in comparision with standard tuples, but that’s pretty much it. You can get the field names with the ._fields function.

Unpacking the dict from the json file into a nametuple is pretty cool, so I include the code snippet here – perhaps some day I will need it 😛

{
    "name": "Stefan",
    "age": 47,
    "likes_python": true
}

import json
from collections import namedtuple

cfg = {}
with open('config.json', r) as cfg:
    cfg = json.load(cfg)
cfg_tuple = namedtuple('X', cfg.keys())(*cfg.values())  # 'X' is the type
print(f'name: {cfg_tuple.name}, age: {cfg_tuple.age}')  # access by field names

name, age, likes_python = cfg_tuple  # unpacking like regular tuple

>>> name: Stefan, age: 47

Hope this helps and let me know what you think about this!