May 7, 2015 - Using Librarian with Katello

Currently, Katello doesn’t handle dependency management for Puppet modules, but you can use librarian-puppet for this purpose.

Let’s create a Puppetfile for our Content View:

forge "https://forgeapi.puppetlabs.com"

mod 'puppetlabs-apache'
mod 'puppetlabs-ntp'

Use librarian-puppet package to get the .tar.gz packaged modules:

# librarian-puppet package

At this point, you’ll have a directory vendor/puppet/cache with all of the puppet modules including their dependencies.

vendor
└── puppet
    ├── cache
    │   ├── puppetlabs-apache-1.4.1.tar.gz
    │   ├── puppetlabs-concat-1.2.1.tar.gz
    │   ├── puppetlabs-ntp-3.3.0.tar.gz
    │   └── puppetlabs-stdlib-4.6.0.tar.gz
    └── source

Upload the modules with hammer:

# hammer repository upload-content --organization="BitBin"\
  --name "Local Forge" --product "Puppet Modules" --path vendor/puppet/cache


[Foreman] Username: admin
[Foreman] Password for admin: 
Successfully uploaded file 'theforeman-dns-2.0.1.tar.gz'.
Successfully uploaded file 'puppetlabs-concat-1.2.1.tar.gz'.
Successfully uploaded file 'puppetlabs-stdlib-4.6.0.tar.gz'.
Successfully uploaded file 'puppetlabs-apache-1.4.1.tar.gz'.
Successfully uploaded file 'theforeman-concat_native-1.4.0.tar.gz'.
Successfully uploaded file 'puppetlabs-ntp-3.3.0.tar.gz'.

But, you’ll probably want to add them to a Content View, too. This will require some fancy dancing with bash, but it appears hammer puppet-module is broken. See: http://projects.theforeman.org/issues/10410

If hammer puppet-module did work, some hacky shell script like this would do it:

#!/bin/bash

USER='admin'
PASSWORD='changeme'

ORGANIZATION=2
REPOSITORY=42

TARGET_CONTENT_VIEW=96

IFS=$'\n'
MODULES=$(librarian-puppet show)

for module in $MODULES
do
  echo ----- $module
  name=$(echo $module | cut -f1 -d\()
  version=$(echo $module | cut -d\( -f2 | cut -d\) -f1)

  module_id=$(hammer -u $USER -p $PASSWORD puppet-module list --repository-id=$REPOSITORY | grep $name | grep $version | cut -d\| -f1)
  hammer -u $USER -p $PASSWORD content-view puppet-module add --organization-id=$ORGANIZATION --content-view-id=$TARGET_CONTENT_VIEW --id=$module_id
done

Jan 6, 2015 - Legal Television and Movies for Expats, Part 2

A while ago,I wrote a blog post about legal television and movies for expats, using streaming services combined with a VPN.

Unfortunately it looks like things are changing: movie studios are fighting back against our outrageous attempt to give them our money, by blocking these VPN services.

No doubt, if this escalates, they’ll certainly find the range my current VPN provider uses and block it, and I’ll be stuck with what’s available in Germany. Thankfully, the situation has improved a lot - Netflix and Amazon both stream in Germany now, but still mostly crappy dubbed content. A very limited selection of Original Version (OV) content is available that falls far short of what’s streamable in the U.S.

Most television shows take years to become available here due to complex localization and licensing requirements. Movies take less time, but still not on par with the U.S., Sony completely ignored the rest of the world when they launched The Interview, for example.

The internet has no borders. Please stop building artificial ones. :-(

Nov 11, 2013 - Foreman FreeIPA Integration Guide


Note! In Foreman 1.5, FreeIPA realm join integration is now built-in!


Two projects that I'm really loving at the moment are The Foreman and FreeIPA. The Foreman is lifecycle management tool for physical and virtual servers (think Cobbler on PCP), and FreeIPA provides central authentication: directory services, kerberos, policy enforcement, and a PKI infrastructure. Why not glue them together? This is my first attempt, and it's all a bit manual and unpolished. There's an effort to get this integration into the Foreman Smart Proxy itself.

All of the scripts here are in a Github Gist.

The goals here are:
  • Foreman authenticates against FreeIPA
  • Signed Certificates for Foreman, Puppetmaster, and Clients
  • New Hosts automatically register to IPA and get a signed certificate
  • A host gets deleted from IPA when it is deleted from Foreman

Prerequisites

  • Installed Foreman Server on a Red Hat-based distro (RHEL, CentOS, Fedora, et al).
  • Foreman server is registered to FreeIPA
  • Installed FreeIPA (or Red Hat IdM) Server

Contents

  1. Foreman LDAP Authentication
  2. FreeIPA PKI Infrastructure
  3. Automatic IPA Registration/Deletion

Foreman LDAP Authentication

End Goal: Users can login to the Foreman using FreeIPA credentials. Individual access rights still need to be granted in the Foreman GUI itself, though. Based on the FreeIPA Ejabberd Integration Guide
  1. Create a foreman.ldif file, replacing dc=bitbin,dc=de with your DN, and providing an appropriately secure password:
    dn: uid=foreman,cn=sysaccounts,cn=etc,dc=bitbin,dc=de
    changetype: add
    objectclass: account
    objectclass: simplesecurityobject
    uid: foreman
    userPassword: 8j926SEpcOvM0WLI
    passwordExpirationTime: 20380119031407Z
    nsIdleTimeout: 0
  2. Import the LDIF (change localhost to an IPA server if needed), you'll be prompted for your Directory Manager password:
    # ldapmodify -h localhost -p 389 -x -D \
    "cn=Directory Manager" -W -f foreman.ldif
  3. Add an IPA group for foreman_users (optional):
    # ipa group-add --desc="Foreman Users" foreman_users
  4. Now login to the Foreman as an Admin, click on "LDAP Authentication" under More/Users. Then click New LDAP Source and fill in the details, changing dn's where appropriate to your own domain:

    • Server: astriaporta.bitbin.de
    • Port: 636
    • TLS: checked
    • Account username: uid=foreman,cn=sysaccounts,cn=etc,dc=bitbin,dc=de
    • Account password: 8j926SEpcOvM0WLI
    • Base DN: cn=accounts,dc=bitbin,dc=de
    • Filter (optional): (memberOf=cn=foreman_users,cn=groups,cn=accounts,dc=bitbin,dc=de)
    • Automatically create accounts in the Foreman: checked
    • LDAP mappings are as the examples given.
    </li> </ol>

    FreeIPA Certs for Foreman + Puppet

    End Goal: Foreman and the Puppetmaster use certificates from the FreeIPA server.
    1. Backup the existing SSL directory:
      # mv /var/lib/puppet/ssl /var/lib/puppet/ssl.old
    2. Make the appropriate directory structure:
      # mkdir -p /var/lib/puppet/ssl/{private_keys,certs}
    3. For the next steps you need to have a Kerberos ticket for a user with sufficient privileges (e.g. admin)
      # kinit admin
    4. Create the service principal:
      # ipa service-add puppet/`hostname`
    5. Request certificates from the IPA server
      # ipa-getcert request -K puppet/`hostname` -D `hostname` \
       -k /var/lib/puppet/ssl/private_keys/`hostname`.pem \
       -f /var/lib/puppet/ssl/certs/`hostname`.pem
    6. Check on the request, you should see the status as MONITORING if successful:
      # ipa-getcert list
      Request ID '20131106211000':
          status: MONITORING
          stuck: no
          key pair storage: type=FILE,location='/var/lib/puppet/ssl/private/gatebuilder.bitbin.de.pem'
          certificate: type=FILE,location='/var/lib/puppet/ssl/certs/gatebuilder.bitbin.de.pem'
          CA: IPA
          issuer: CN=Certificate Authority,O=BITBIN.DE
          subject: CN=gatebuilder.bitbin.de,O=BITBIN.DE
          expires: 2015-11-07 21:10:01 UTC
          eku: id-kp-serverAuth,id-kp-clientAuth
          pre-save command:
          post-save command:
          track: yes
          auto-renew: yes
    7. Copy the IPA CA.cert:
      # cp /etc/ipa/ca.crt /var/lib/puppet/ssl/certs/ca.pem
    8. Take a peek in the SSL directories, and you'll see our new certs:
      # ls  {private_keys,certs}
      certs:
      ca.pem  gatebuilder.bitbin.de.pem
      
      private_keys:
      gatebuilder.bitbin.de.pem
    9. Make sure permissions are sensible:
      # chown -R puppet:puppet /var/lib/puppet/ssl
      # chmod 600 /var/lib/puppet/ssl/{private_keys,certs}/`hostname`.pem
    10. Edit /etc/puppet/puppet.conf:
      • Add to [main]:
        # This disables the CRL.  I need to fix this at a
        # later time
        certificate_revocation = false
      • In [master], change:
        ca = false
      • Restart httpd (Foreman-configured Puppet runs in Passenger):
        # service httpd restart
    11. Browse to the Foreman, and you should see it using the new SSL certificates signed by your IPA CA. Ideally you should import the IPA CA.crt on your local box and trust it.
    12. </ol>

      Registration at Provision-Time

      The idea here is that our machines when foreman creates them are automatically registered to FreeIPA with a one-time password, and if later deleted in the Foreman, they are removed from FreeIPA too. Hosts also get an SSL certificate signed by the FreeIPA server to talk to puppet. The flow looks like this: Here's a really confusing graphic that may or may not make things clearer

      Creating IPA User with Right Permissions

      A previous version of this guide called this user "foreman" - don't do that, it'll interfere with upgrading later, as the RPM packaging expects to use a local user named "foreman."
      1. Create the user:
        # kinit admin
        Password for admin@BITBIN.DE:
        # ipa user-add --first="The" --last="Foreman" foreman_reg \
        --password
        Password:
        Enter Password again to verify:
        --------------------
        Added user "foreman_reg"
        --------------------
      2. Grant host enrollment privileges:
        # ipa role-add-member --users=foreman_reg "Host Enrollment"
      3. We need to modify the Host Enrollment role to actually allow the Foreman user to add brand new hosts and delete them too -- so Foreman can completely manage the machine lifecycle.
        # ipa privilege-add-permission 'Host Enrollment' \
        --permissions='Add Hosts'
        # ipa privilege-add-permission 'Host Enrollment' \
        --permissions='Remove Hosts'
      4. Change foreman password after first time:
        [root@gatebuilder ~]# kinit foreman_reg
        Password for foreman_reg@BITBIN.DE:
        Password expired.  You must change it now.
        Enter new password:
        Enter it again:
        </ol>

        Configuring the Create/Destroy Hook

        Grab the scripts from the Github Gist.
        1. On Foreman, install the hooks gem:
          # yum -y install ruby193-rubygem-foreman_hooks
        2. And make the directory structure we need:
          # mkdir -p /usr/share/foreman/config/hooks\
          /host/managed/{create,destroy,after_commit}
        3. Put foreman-ipa into /etc/sysconfig/ with the right permissions
          # chown foreman /etc/sysconfig/foreman-ipa
          # chmod 600 /etc/sysconfig/foreman-ipa
        4. Configure /etc/sysconfig/foreman-ipa
          # Are we using IPA as the CA?
          CREATE_SERVICE_PRINCIPAL=true
          
          # Allow Foreman to delete hosts from IPA
          PREVENT_DELETING_HOSTS=false
          
          # Hostname of an IPA server
          IPA_SERVER="astriaporta.bitbin.de"
          
          # User with appropriate permissions
          IPA_USER="registration"
          IPA_PASS="password"
          
          # Foreman API User/Password
          FOREMAN_USER="apiuser"
          FOREMAN_PASS="apipass"
        5. Put 10_integrate_freeipa.sh into /usr/share/foreman/config/hooks/host/managed/create and create a symlink to destroy and after_commit:
          # ln -s /usr/share/foreman/config/hooks\
          /host/managed/create/10_integrate_freeipa.sh ../destroy
          # ln -s /usr/share/foreman/config/hooks\
          /host/managed/create/10_integrate_freeipa.sh ../after_commit
        6. Restart foreman to get it to notice the new hooks:
          # service foreman restart
          # service httpd restart
        7. Take a look at the logs to make sure the hooks were registered, look in /var/log/foreman/production.log:
          Finished registering 1 hooks for Host::Managed#destroy
          Finished registering 1 hooks for Host::Managed#after_commit
          Finished registering 1 hooks for Host::Managed#create
        8. The last step in this is integrating into your provisioning template. You'll need to get ipa-client installed in your packages list, and remove the other puppet registration thingy from the Foreman. I have a snippet that looks like this:
          # Register to IPA, two times
          # in case of https://fedorahosted.org/freeipa/ticket/3377
          ipa-client-install --mkhomedir -w <%= @host.params['ipa_onetime'] %> -f -U
          ipa-client-install --mkhomedir -w <%= @host.params['ipa_onetime'] %> -f -U
          
          # Make Puppet Certificate Directories
          mkdir -p /var/lib/puppet/ssl/{private_keys,certs}
          
          # Generate IPA Certificate
          ipa-getcert request -K puppet/<%= @host.name %>  -D <%= @host.name %> \
          -k /var/lib/puppet/ssl/private_keys/<%= @host.name %>.pem \
          -f /var/lib/puppet/ssl/certs/<%= @host.name %>.pem
          
          # Workaround for "stack too deep" problem
          # http://projects.puppetlabs.com/issues/21869
          cp /etc/ipa/ca.crt /var/lib/puppet/ssl/certs/ca.pem
          
          cat <<EOF > /etc/puppet/puppet.conf
          [main]
              # The Puppet log directory.
              # The default value is '$vardir/log'.
              logdir = /var/log/puppet
          
              # Where Puppet PID files are kept.
              # The default value is '$vardir/run'.
              rundir = /var/run/puppet
              ssldir = /var/lib/puppet/ssl
              server = <%= @host.puppetmaster %>
          
          [agent]
              # The file in which puppetd stores a list of the classes
              # associated with the retrieved configuratiion.  Can be loaded in
              # the separate ``puppet`` executable using the ``--loadclasses``
              # option.
              # The default value is '$confdir/classes.txt'.
              classfile = $vardir/classes.txt
          
              # Where puppetd caches the local configuration.  An
              # extension indicating the cache format is added automatically.
              # The default value is '$confdir/localconfig'.
              localconfig = $vardir/localconfig
          
              certificate_revocation = false
              certname = <%= @host.name %>
          EOF
          
          puppet agent --test
          chkconfig puppet on
        9. When the machine boots it will use one-time password authentication with FreeIPA and grab an SSL certificate for use with Puppet. And you get the bonus of when you delete the machine in Foreman, it gets deleted in IPA too.
        10. </ol>