There has been a sea-change in the worlds of MariaDB and MySQL. They have, belatedly but still in a most welcome fashion, come around to the way of thinking that has been the norm in the daemontools world for two decades. Version 1.28 of the nosh toolset takes advantage of this thinking to bring service bundles for MariaDB and MySQL closer to the daemontools norm.
The (very) old way of doing things was for MariaDB and MySQL to be started by an
rc script, for either van Smoorenburg
rc (on Linux operating systems of various flavours) or Mewburn
rc (on FreeBSD, TrueOS, DragonFly BSD, NetBSD, and so forth).
MariaDB directly included a script for Debian-flavour van Smoorenbug
Both MariaDB and MySQL directly included an attempt at a more generic van Smoorenburg
rc script, named
mysql.server (as documented by Oracle for use in van Smoorenburg
The FreeBSD (and NetBSD) "ports" tree contained the Mewburn
rc script for MySQL, named
In both the Mewburn and van Smoorenburg
rc systems the scripts would invoke the database server indirectly, via another script named
They would read their service configuration information from an operating-system-specific expected place:
/etc/rc.conf in the case of the BSDs; a messy choice between
/etc/conf.d in the case of the various van Smoorenburg
This was a rickety, byzantine, and unsafe mess.
It demanded that a "PID file" be used (a notoriously unsafe mechanism that we've known to avoid since the 1990s), and gave three different places where it might be set: a
mysql_pidfile setting in
pid-file setting in
/etc/mysql/my.cnf, or perhaps
/etc/my.cnf, depending), or a
mysqld_pid_file_path setting in one of the multiplicity of van Smoorenburg flavours' system configuration files.
This necessitated running the
my_print_defaults command, to read these options.
This in turn necessitated finding out where
my_print_defaults and the
my.cnf file were.
mysqld_safe script that was invoked went and duplicated these same steps.
Things only got worse from there, more on which later.
In versions 1.27 and earlier of the nosh toolset. there was a pre-supplied service bundle to run either MariaDB or MySQL, in
The external configuration import subsystem would (say) import
mysql_enable and the other
mysql_XYZ into the nosh service management's enable setting and the XYZ variables set in that service bundle.
The administrator's experience would thus be fairly close on a nosh-managed system to a van Smoorenburg or Mewburn
After importing the configuration into the nosh service bundles, an administrator could still run
service mysql start to start the database service,
initctl mysql status to see its status, and so forth, using the various shims that map these onto native nosh mechanisms.
There were still a few rough edges, though.
Looking more closely at
mysqld_safe one finds that invoking it under systemd, or indeed any decent service manager, would qualify it as a candidate for the systemd House of Horror.
mysqld_safe is that classic of the systemd House of Horror: a Poor Man's dæmon supervisor written badly in shell script (as they always are).
Consider how it operates:
It tries to work out where the
my_print_defaults program is.
my_print_defaults program, it parses
my.cnf (from wherever it is located) to find the settings in the
[mysqld_safe] section; translating them into command-line arguments for itself.
It parses those command-line arguments, alongside any that it has been explicitly given. These command line arguments specify things like the location of a "PID file", the user account to run the server process as, process priority settings, open file descriptor limits, core file size limits, NUMA control, and auto-restart settings.
It works out a sequence of chaining commands that applies all of the limits, accounts, priority settings, and so forth.
It enters a loop, spawning an associated log dæmon and the MariaDB/MySQL server dæmon (connected via a pipe), waiting for the pipeline to terminate, and executing the restart logic.
The MariaDB/MySQL server is run via the chain of commands calculated earlier on, as a child process of the
People from the daemontools world will readily observe that this attempts to duplicate the logic of proper service management in shell script, but (as is ever the case with such Poor Man's service managers) does so badly.
The pipe isn't held open by the manager process, so data can be lost if the log dæmon fails for some reason. This is a well-known design feature of daemontools, that has been around since 1997.
The log dæmon isn't independently restartable. Neither is the main dæmon. Both dæmons have to shut down in order for either to be restarted, because of course a (job control) shell operates in terms of entire pipelines. Independent restart has also been the daemontools norm for two decades.
Service managers expect the process that they spawn to be the dæmon.
They expect to be able to send
KILL and other signals to that process and have them affect the actual service.
The log dæmon isn't replaceable with a different style of log handler according to the system administrator's need/taste.
It has a lot of "PID file"-style logic, including the wacky idea of stray dæmons that are killed with the
KILL signal because they have the same name as the server program.
Needless to say, this is incompatible with the notion of running multiple database servers, pointing at different directories but using the same server program images.
Yet more unsafe PID file logic is added to bodge around this.
It is a completely unnecessary layer.
Users of daemontools, runit, and other such service managers have been pointing out for a decade and a half that this is completely wrongheaded, and is doing badly in an idiosyncratic service-specific script the job that general-purpose service management itself does well. They have long pushed for ways to run MySQL and MariaDB properly, with all of this stuff done by the actual service management mechanism. Here are just a few examples:
In 2002, Tatsuhiko Miyagawa wanted to know about just invoking the
mysqld program directly from a daemontools
In 2003, Erich Schubert filed Debian bug #208364 asking that MySQL be properly manageable without the multiple layers of van Smoorenburg
rc script nonsense.
In 2010, Alex Efros asked, in MySQL bug #57690 that the MySQL server just log to standard output/error, so that xe could attach a separately managed log dæmon of xyr choosing, without MySQL making wholly unwarranted assumptions about its standard output/error being a file.
In 2009 Mario Limonciello of Ubuntu took the same direction, writing a
mysql.conf job file for upstart that set the process limits and restart controls using upstart's own general-purpose native mechanisms and invoking
Six years later, Robie Basak observed that "under no circumstances should the code in the init.d script actually run on an upstart system".
The thinking from the daemontools world was, after almost fifteen years, finally catching on.
The comparatively highly belated prompt for the change that came from systemd can be found in MySQL bug #65809 filed in 2012 by Honza Horak of RedHat, who pointed out that
mysqld_safe should be done away with in 2014.
In the MariaDB world this continued with MariaDB development task #5713, and has culminated with improvements to how one starts the MariaDB dæmon under a service manager that are (unfortunately) to be found under the heading "systemd" in the MariaDB doco (rather than in a more general place applicable to all modern service managers from the 1990s onwards). These improvements started to manifest in mid-to-late 2015 with work such as this change by Daniel Black of IBM, and open discussions such as MariaDB PR #26 and MariaDB PR #83.
Oracle's response to the original bug was churlish and uninformative, in comparison. M. Horak's three-year-old bug report was simply closed after pretty much total silence on the matter with a curt "contribution rejected" and a pointer to where Oracle supposedly has done the work without saying what work has been done. Unlike MariaDB, there's no open discussion detailing the design decisions nor detailed documentation of the revised way of doing things.
In the systemd and upstart worlds,
mysqld_safe is now not the way to start the servers.
All of the things that it does are now done by the service management mechanisms, doing things the way that the daemontools world has pushed for all along.
The dæmon run is
mysqld itself, not via wrapping layers of scripts upon scripts.
Moreover: it is run, not spawned.
The process that service management supervises is the server dæmon itself.
mysqld writes its logs to standard output/error.
With systemd service management that goes to the systemd journal.
With upstart service management that goes to whatever logging has been configured with the
With daemontools family service management that goes to whatever program is being used in the dedicated log dæmon.
Process limits, priorities, user accounts, and suchlike are controlled with native service manager mechanisms.
Service management does the (independent) auto-restart of server and log dæmons, and keeps the pipe between them open across restarts.
In addition, in the systemd world there is now no longer a single fixed service unit defining the service — at least, for MariaDB.
Instead, the package installation procedure runs a
mariadb-service-convert command that reads
my.cnf (once again using
my_print_defaults) and writes out an override file for
mariadb.service with all of the process limits, accounts, priorities, and suchlike turned into the native systemd mechanisms.
This more closely parallels the way that nosh service management operates with other services.
In nosh version 1.28, the
/var/sv/mysql service bundle is now simply an alias.
The external configuration import subsystem, whenever it is told to update its imported configuration, finds the MariaDB/MySQL configuration file and instantiates one or more nosh service bundles, containing all of the process limit, user account, priority, NUMA control, and suchlike dæmon configuration options converted to nosh native mechanisms.
One bonus feature of doing things this way lies in "one or more".
The nosh external configuration import subsystem recognizes the multiple-instance conventions employed by MariaDB (multiple
/etc/mysql/my*.cnf files) and MySQL (multiple
] sections in a single
my.cnf file) and generates multiple nosh service bundles, with their own individual settings and logging.
The PostgreSQL world lags the MariaDB and MySQL worlds when it comes to doing things the way that they've been done with daemontools family service management for two decades. However, in 2015 Peter Eisentraut proposed doing away with PostgreSQL's idiosyncratic log management and just sending logs to standard error.