Developing/Testing Custom Facts for Puppet

An important part of any puppet deployment are custom facts. Facts identify specific attributes about the machines in your environment. For example, a fact might indicate if a specific application is installed, another may identify the version of that application. Some facts are built in and available by default, for example OS information. Other facts are too specific to your environment and facts must be created and deployed. One of the best ways to create facts in puppet is to use custom facts. These facts are written in Ruby scripts using the Facter API.

I am a senior architect at a SaaS software company. I oversee the monitoring of all of our production hardware, database and other equipment, virtual and physical. We use puppet to automatically deploy all monitoring technology. As soon as new machines register with puppet (part of the initial image), our custom facts run. These facts are used to determine what needs to be monitored, what needs to be deployed to the new machine and what configuration needs to be updated in other places.

Structure of a Custom Fact in Ruby

Custom facts have a specific structure. Adherence to the structure is important as the whole fact does not necessarily execute all at once, or even in a single context. The structure is as follows

Facter.add(<fact name>) do
   confine :<someFact> => '<some value>'
   confine :<someOtherFact> do |f|
       f == '<value 1> || f == '<value 2>'
   end
   has_weight <some number>
   setcode do
      # code for fact ...  
   end
end

The puppet help documentation explains how to write a custom fact pretty well. The documentation is available here. There are a couple of key things to note which are easy to miss.

Firstly, there can be multiple confine statements. Confine statements are actually a block, and that is how you can confine the fact to any of multiple possible values. It is documented, but easy to miss initially.

Secondly, it is important to realize that the setcode block does not necessarily run when the Facter.add() block runs. Instead it is possible for puppet to take a reference to the setcode block and execute it later. This important, as sometimes, there are multiple implementations for the same fact, and puppet has to process all of them and determine which one applies. Puppet tries to only execute the setcode block in the fact that has the highest weighting. Any code which is outside the confine blocks and outside the setcode block will run, but the code in the setcode block might run in a totally different context and none of the outer results may be available to it. The only code outside a setcode or confine block should be the has_weight directives.

How to Test Run a Custom Fact

This is something that was not well documented or obvious. Initially, for me, testing Custom facts involved adding it to a module or manifest, ensuring that it only got deployed to a test machine and then letting it execute as part of the normal 30 minute check-in cycle. Of course, you can log into the remote machine to force it to check in immediately with

$ puppet agent -vt

on the specific host. However, that is a slow process and runs everything associated with the machine. In some environments this can take a few minutes. Then you have to determine if the fact has the intended value, or a value at all.

If only there was an easier way? It is not possible to simply run the ruby code using the system ruby. Custom facts are written in a Ruby based DSL (domain specific language). The confine, setcode and has_weight are not native to ruby at all. It is even worse if you are trying to fix and existing custom fact, or extend it, as it is already deployed. It is dangerous to try and edit and hope for the best. Just to make matters trickier, Puppet does not necessarily use the system version of ruby. Puppet typically embeds a more recent version of ruby in its deployment.

There is a simple way to test run a custom fact. It is as follows,

$ FACTERLIB=<directory of fact> facter <name of custom fact>

or

$ facter --custom-dir=<directory of fact> <name of custom fact>

For example,

$ facter --custom-dir=. foobar

This will execute facts in the current directory and display the value of the fact called foobar after. This is huge, it does not take long to execute. You do not need to do anything to deploy the fact, you can simply copy it to the machine you want to test on. It does not mess with anything else. The FACTERLIB environment variable above is actually just a path specification, so it can contain multiple colon separated paths.

Another huge advantage of testing facts like this is that print statements can be used to output values when developing the fact. When custom facts are run via puppet, the printed output is lost. This can help with rapidly developing or debugging facts.