Ameet Paranjape

The bread and butter of Landscape is security compliance – keeping your systems securely up to date with minimum effort, and remedy any unintended side-effects an update may have.  Helping your CIO prove this was done timely is a neat additional capability that can save the IT team the need to scrub logs to create reports.

“Help me keep my Ubuntu systems compliant with the latest security updates” is a common request I hear from frustrated System Administrators.  Landscape was designed with compliance in mind from the start, but since I have explained and demonstrated this countless times to many customers in person, I thought I’d share the benefits of managing updates through Landscape with a larger audience.

The “freshness” of Ubuntu, this wonderful ability to find any and all software, in the latest version, always readily packaged for your favorite distribution, comes with a constant stream of updates that need to be managed.

To start, let’s first approach this using a reactive method and finish with a proactive approach that will keep our systems compliant going forward.

In Landscape, select the “COMPUTERS” tab.

 Under Select Computers choose “All” as shown below.  In my case, I have 5 servers but you could have any number of physical servers, desktops,  virtual machines or cloud  guests under management in your account.

Select the “Packages” tab on the right:

In the Summary section of my example, I have “3 computers with security updates to install” link.  You could have hundreds, thousands or even tens of thousands of managed systems – the strength of Landscape’s interface is how it makes it as simple to manage a thousand systems as it is to manage one.

Landscape lets you easily distinguish between security updates, and feature updates and bug fixes – so if you desire to reduce the amount of changes you push to your systems, or to schedule a different maintenance event for less urgent updates, this is easily done.

Select the link to get a complete list of security fixes and select “All upgrades.” This will toggle the install selection of all the security updates listed – Landscape lets you select a different behavior for each of the security updates listed, from the same overview screen. One could, for example, select to uninstall cups altogether if the vulnerable systems do not support any actual printing.  This is done through the three buttons on the left.

The selected action is reflected in the color change:

Scroll to the bottom of the screen and select Apply Changes.  You can apply the updates right away or select a future date and time. As an aside, I strongly recommend using Landscape’s system tagging feature for rolling updates and avoiding regressions, by grouping similar systems together based on their workloads

.

At this point all our systems will have the latest Ubuntu security updates installed shortly.

Now let’s examine how to keep systems current proactively by using Landscape’s package profiles feature. From the Landscape tab,  select Profiles.

In the next window, select Upgrade Profiles.

Click on “Add update profile” and enter in the required fields.  The title of this profile is “Ubuntu Security Updates” but you could name your profile whatever you want.  We have checked the “only security updates” box so this profile won’t apply feature updates or bugfixes, and requested the packages be applied on all systems at 2am every Sunday.  Click the Save button and it will take effect.

Profiles are editable, and you can combine profiles with Landscape’s system tags for granular control of the update process.  A common example is using different scheduled update profiles for roaming devices: you could mandate after-hours updates for the laptops of your sales team to enforce your security policy, yet make sure that no lingering Landscape tasks interfere with their work hours if the systems were offline or powered off overnight.

Sometimes, an update does not go exactly as planned.  Packages are hardly ever defective on their own, but sometimes an update may be having an unintended side effect on your workload, and you need to roll back the patch you just installed to continue operating until a more complete or fully functional solution can be devised.

Landscape makes this remarkably easy. Searching for and selecting the offending package is done in the packages tab, and expanding the details section will show what managed devices have the package installed, and what versions it can be reverted to.  Once you are ready to commit, the downgrade process is just as streamlined as the upgrade.

Many of our customers have Compliance or Governance requirements: they need to prove to an oversight body, either internally to their company or externally to a certification authority, that systems were kept secure.  The most common example of this is the PCI DSS rule requiring security updates be rolled out within 30 days of release.

You can help your CIO fulfill these requirements effortlessly, by providing her with a CSV or a chart showing the current state of update of your infrastructure, generated using Landscape’s reporting functionality:

Even when no security audits have to be met, folks are pretty happy when you show a high percentage of systems are securely patched  and you can justify the groups of systems that have yet to be covered.  Auditing is awesome, as it generates CSV data and charts without requiring you to spend any cycles sifting through logs gathering details.

This is just one of the many compliance options available in Landscape.  You can watch a four-minute video overview on YouTube,  or Check out Landscape yourself.

Federico Lucifredi

Today we’re introducing some new features into Ubuntu’s systems management and monitoring tool, Landscape. Organisations will now be able to use Landscape to manage Hyperscale environments ranging from ARM to x86 low-power designs, adding to Landscape’s existing coverage of Ubuntu in the cloud, data centre server, and desktop environments. There’s an update to the Dedicated Server too, bringing SAAS and Dedicated Server versions in alignment.

Calxeda 'Serial Number 0' in Canonical's lab

Hyperscale is set to address today’s infrastructure challenges by providing compute capacity with less power for lower cost. Canonical is at the forefront of the trend. Ubuntu already powers scale-out workloads on a new wave of low-cost ultradense hardware based on x86 and ARM processors including Calxeda EnergyCore and Intel Atom designs. Ubuntu is also the default OS for HP’s Project Moonshot servers.

Calxeda 01 - Wide

This update includes support for ARM processors and allows organisations to manage thousands of Hyperscale machines as easily as one, making it more cost-effective to run growing networks spanning tens of thousands of devices. The same patch management and compliance features are available for ARM as they are for x86 environments, making Landscape the first systems management tool of a leading Linux vendor to introduce ARM support – and we are doing so on a level of feature parity across architectures.

Calxeda is the leading innovator engaged in bringing ARM chips to servers and partnered with us early on to bring Ubuntu to their new platform. “Landscape system management support for ARM is a huge step forward”, said Larry Wikelius, co-founder and Vice President at Calxeda. “Adding datacenter-class management to the Ubuntu platform for ARM demonstrates Canonical’s commitment to innovation for Hyperscale customers, who are looking to Calxeda to help improve their power efficiency.”

Calxeda 'Serial number 0' in Canonical's Boston lab

“Landscape’s support for the ARM architecture extends to all ARM SoCs supported by Ubuntu, but we adopted the Calxeda EnergyCore systems in our labs as the reference design in light of both their early arrival to market and their maturity”, said Federico Lucifredi, Product Manager for Landscape and Ubuntu Server at Canonical, adding “we are excited to be bringing Landscape to Hyperscale systems on both ARM and x86 Atom architectures.” CIOs and System Administrators considering implementing Hyperscale environments on Ubuntu will now have access to the same enterprise-grade systems management and monitoring capabilities they enjoy in their data centres today with Landscape.

Kurt Keville, HPC Researcher at Massachusetts Institute of Technology (MIT) commented: “MIT’s interest in low power computing designs aims to achieve the execution of production HPC codes at the same level of numerical performance, yet within a smaller power envelope.”  He added: “With Landscape, we can manage our ARM development clusters with the same kind of granularity we are accustomed to on x86 systems. We are able to manage ARM compute clusters without affecting our production network bandwidth in any way”.

Parallella Gen0 prototypes stack

The Parallella Board project aims to make parallel computing ubiquitous through an affordable Open Hardware platform equipped with Open Source tools. Andreas Olofsson, CEO, Adapteva said: “We selected Ubuntu as our default platform because of its popularity with the developer Community and relentless pace of updating, regularly providing our users with the newest builds for any project.”  He added: “ The availability of a management and monitoring platform like Landscape is essential to managing complexity as the scale of Parallella clusters rapidly reaches into the hundreds or even thousands of nodes.”

Parallella 01 - Processes

As we talk to customers building cloud infrastructure or big data computing environments, it’s clear that power consumption and efficient scaling are key drivers to their architectural decisions. When these considerations are coupled with Landscape’s efficiency and scalable management characteristics, we believe enterprises will be able to achieve a significant shift in both scalability and manageability in their data centre through Hyperscale architecture.

Ubuntu is the default OS for HP’s project Moonshot cartridges, ships or is available for download to every Moonshot customer, with direct support from HP backed by Canonical’s worldwide support organization.  The Landscape update today also means that the full bundle of Ubuntu Advantage support and services becomes available to Moonshot customers.

“Canonical continues to lead the way in the Hyperscale OS arena introducing full enterprise-grade support services for Ubuntu on Hyperscale hardware”, remarked Martin Stadtler, Director of Support Services at Canonical.

Landscape’s Dedicated Server edition has also been refreshed in this update. This means that those businesses choosing to keep the service onsite (rather than hosted) will benefit from the same functionality and a series of updates already available to SAAS customers, including the new audit log facility and performance enhancements, while retaining full local control of their management infrastructure.

Mike Milner

In our last blog on Landscape and Puppet, we talked about using Landscape to automatically deploy Puppet on new computers. In this article we’ll dive deeper, and look at how to use Landscape as an External Node Classifier for Puppet.

In a typical Puppet configuration, all your nodes are defined in site.pp under the “node” definitions section. Each node is assigned Puppet classes by manually editing the file. When your number of nodes grows, manually managing the site.pp file becomes tedious and error prone.

An External Node Classifier (ENC) is Puppet’s way of offloading the tedious node maintenance to an external program. The interface is dead simple – puppet executes the external node classifier program with a single full node name as the only argument. The classifier just has to write a YAML blob out to stdout before exiting.

To start, let’s create a simple python ENC in /etc/puppet/landscape_enc – Don’t forget to make the file executable.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
#!/usr/bin/env python
import sys
import yaml

# The node name to be classified is supplied as an argument to the script
node_name = sys.argv[1]

classes = ["basenode"]

# Output must be a YAML document
print(yaml.dump({
    "classes": classes,
    }))

 

It ignores the node name and just puts everything into the “basenode” class. Not very interesting but it’s enough to get started with Puppet.

NOTE: These examples are all using puppet 2.7 which ships on Ubuntu 12.04 Precise LTS. The ENC functionality behaves a bit differently in versions of puppet earlier than 2.65 – See http://docs.puppetlabs.com/guides/external_nodes.html for details.

To test the ENC I put together a minimal puppet configuration with two simple classes and put everything into my site.pp in /etc/puppet/manifests/site.pp

1
2
3
4
5
6
7
class basenode {
  notify {"I am a basenode!":}
}

class specialnode {
  notify {"I am a specialnode!":}
}

 

Notice that no nodes are actually defined. That is the ENC’s job. To enable the ENC you need to add two lines to your puppetmaster’s config file /etc/puppet/puppet.conf

Add these lines at the end of the “[master]” section:

1
2
node_terminus = exec
external_nodes = /etc/puppet/landscape_enc

 

You can now test that the puppetmaster is using your new, rather basic, ENC.

1
2
3
4
5
6
7
ubuntu@ubuntu:~$ sudo puppet agent --test

info: Caching catalog for ubuntu
info: Applying configuration version '1354824718'
notice: I am a basenode!
notice: /Stage[main]/Basenode/Notify[I am a basenode!]/message: defined 'message' as 'I am a basenode!'
notice: Finished catalog run in 0.06 seconds

 

As you can see, with our trivial ENC everyone is a basenode.

Now we’re going to enhance our ENC to ask Landscape to classify nodes for us.

The Landscape API
To use the Landscape API you need three pieces of information: the Landscape API endpoint URI, the user key, and the secret for your user.

To find your API credentials, log in to the Landscape web interface and click your username on the top right. Your API credentials are in the “API Access” section.

For this example, we’ll use the python API client provided with Landscape (NOTE, you must install the landscape-api package first). Here’s how to query for a computer registered with Landscape using it’s host name:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
from landscape_api.base import API

landscape_uri = "https://landscape.canonical.com/api/"
landscape_key = "43NW6OV71L32CSOPCJGX"
landscape_secret = "agBf3v267DqO8vtVRnzjseWfYdV4ueklj5a81iIT"
api = API(landscape_uri, landscape_key, landscape_secret)

api.get_computers(query="my_hostname_here")
[{u'access_group': u'server',
u'comment': u'',
u'hostname': u'appserv1',
u'id': 1,
u'last_exchange_time': None,
u'last_ping_time': u'2012-09-07T15:19:22Z',
u'reboot_required_flag': False,
u'tags': [u'lucid', u'server', u'puppet-webfarm'],
u'title': u'Application Server 1',
u'total_memory': 1024,
u'total_swap': 1024}]

 

Now if we combine that with our ENC we get the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
#!/usr/bin/env python
import sys
import yaml
from landscape_api.base import API

# Create our connection to Landscape
landscape_uri = "https://landscape.canonical.com/api/"
landscape_key = "43NW6OV71L32CSOPCJGX"
landscape_secret = "agBf3v267DqO8vtVRnzjseWfYdV4ueklj5a81iIT"
api = API(landscape_uri, landscape_key, landscape_secret)

# The node name to be classified is supplied as an argument to the script
node_name = sys.argv[1]

# Ask Landscape about the computer
computers = api.get_computers(query=node_name)

# If we don't get back any computers or if we get more than one, error out.
# You could also handle this case by simply giving the node a default class.
if len(computers) != 1:
    sys.stderr.write("Only expecting one computer, instead got this: %s" % computers)
    sys.exit(1)

# Extract the tags from our computer
tags = computer[0]["tags"]

# Now here you can use whatever logic you want to convert
# tags into classes. I'm going to use any tag that starts with "puppet-"
# as a class name. I'm also going to make sure every node gets the
# "basenode" class
classes = ["basenode"]
for tag in tags:
    if tag.startswith("puppet-"):
        class_name = tag.split("-",1)[1]
        classes.append(class_name)

# Output must be a YAML document
print(yaml.dump({
    "classes": classes,
    }))

 

That’s all there is to it. Now if you tag a computer “puppet-database” in Landscape, it will automatically get the “database” class in Puppet.

You can see in the script comments that it’s very easy to customize the behaviour to match your environment. Now tag away and let Landscape and Puppet take over your world!

Matthew Wedgwood

We are updating Landscape on October 25th. While we roll-out the code to Landscape’s servers, we’ll have to take the service offline.

Goes offline: 14:00 October 25th 2012 UTC
Expected back online: 15:00 October 25th 2012 UTC

We’re sorry for the downtime you’ll experience during the roll-out.

Federico Lucifredi

Mastering Package Holds

Landscape’s new release brings to full circle the changes that began a few months ago, beginning with the deprecating of support for Smart package locks: with full APT integration in the client, we are now exposing dpkg holds as first class citizens.

Package holds exist to stop a package from being updated, the most typical case being the commonplace sudo apt-get upgrade mass-upgrade command. We could use package holds to instruct the update stack not to modify my system’s currently installed version of Python:

echo "python2.7 hold" | sudo dpkg --set-selections

and good old dpkg would oblige.  What we have done here is changing the dpkg policy for this package’s upgrades from the default install to hold, the system does the rest, and stops offering us updates for this package. Similarly, I can set a hold on a package from Landscape, and on a broader selection of our datacenter assets. in this case, let’s select the venerable AWK:

 

[Placing a hold on AWK]

All I had to do was click the package icon on the right and applying the change – in Landscape, doing this for a lone machine or a fleet of thousands takes the same time and effort. In a minute, Landscape confirms successful execution:

[activity results]

 

What’s more, the very same hold Landscape configured is now visible at the shell level on the client system, in a seamless display of integration between dpkg and Landscape – here is the shell view before and after the command was issued:

 

[the shell view]

 

Once a hold is in place for a package, regular system upgrades will not change the installed version, even when a newer version becomes available in the configured repositories – regardless of where the update command was issued from, Apt or Landscape.  This provides administrators with a convenient way to protect themselves from accidental mistakes: when a workload is known to be dependent on a specific package version, holds make the system track and enforce this requirement for you – a more reliable and stress-free approach than post-it notes, spreadsheets, or the shared oral culture of your IT team.

A fairly common scenario exposing additional nuances is holding the kernel version. If a kernel update is made available, two scenarios are possible. In the simplest case, a new build of the kernel is available, with changes that do not impact the kernel ABI, and the installed package name would remain the same – for example, linux-image-3.2.0-23-generic-pae, with version going from 3.2.0-23.35 to 3.2.0-23.36 for the new build. Holding this package name will block minor kernel updates that remain ABI-compatible.

The second scenario is more interesting. When the kernel upgrade involves an ABI change, Ubuntu’s Kernel team will introduce a new package, with a different name, like linux-image-3.2.0-24-generic-pae, and as this is a differently named package, the hold on the previous version will not block this new install. The key to restricting kernel upgrades involving an ABI change is in holding the kernel metapackage upgrade, which is triggering the new install through its dependencies.

On my system, I use the linux-generic-pae kernel flavour. Holding the corresponding metapackage does the trick:

echo "linux-generic-pae hold" | sudo dpkg --set-selections

Holding the metapackage upgrade restrains the system from upgrading to the latest kernel with the new ABI — but will allow ABI-compatible upgrades if any become available.  This is reflected both in upgrade selections managed by apt-get upgrade, and remotely by Landscape, as you can see graphically in Landscape’s interface:

To complete our hold master class, we should point out that holds are designed to block package updates, not new installs. If you instruct dpkg, apt-get or Landscape to specifically install a given package, your instructions shall be obeyed. The system is designed with the assumption that you know what you are doing, and holds are used to manage the update system’s behaviour, not to stand in the way of the admin.

We want the user experience of Landscape’s centralized console to reflect the interaction an administrator has with the terminal of a single Ubuntu system: following a Principle of Least Surprise philosophy in our design, we leverage your existing Ubuntu architectural knowledge as much as possible.  This way, we can reflect the user’s expectation of Ubuntu’s behaviour, and flatten Landscape’s learning curve.  To accomplish this, we rely as much as possible on Ubuntu’s built-in default interfaces to the system’ state, like lshw or dpkg, as data sources and management interfaces for the Landscape client.

And those are package holds. Now just one click away in Landscape.

Mike Milner

Integrating Landscape and Puppet

There is something wonderful about getting new hardware. I love opening all the boxes and putting new servers into racks and plugging everything together. Setting up the OS and software however can be a pain. This is where Puppet and Landscape come to the rescue.

Puppet is great at configuring nodes the way you want them and keeping them that way. On the puppet master, you describe various aspects of the system using a simple declarative language. On the managed computers, a small agent talks to the puppet master and pulls down config changes. If you need to change a configuration on all your machines, you can change it once on the puppet master and it gets pushed out to all the agents.

In this article we’re going to use Landscape to do the initial setup of puppet. Tag a computer in Landscape with “puppet”, and the puppet agent will automatically be installed and configured. We’re going to use a meta-package plus some new Landscape features to make it all easy.

Meta-Package
The meta-package is a common trick used on Debian and Ubuntu based systems to easily pull in a subset of packages. Here we’ll use a meta-package to pull in all the puppet dependencies then run a small script to register the puppet agent to the puppet master.

Our meta-package will only have two files: control, and postinst. These files will both live in the DEBIAN directory like this:

working directory
  |> landscape-puppet
      |> DEBIAN
          |> control
          |> postinst


Here’s the control file:

1
2
3
4
5
6
7
8
Package: landscape-puppet
Version: 0.1
Section: main
Priority: standard
Architecture: all
Depends: landscape-client, puppet
Maintainer: Mike Milner <mike.milner@example.com>
Description: Configure system for use with puppet.


And here’s the postinst file – this gets executed after the package is installed:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
#!/bin/bash

PUPPETCONFIG=/etc/puppet/puppet.conf

# Back up current puppet config if it exists
if [ -f "$PUPPETCONFIG" ]; then
   mv $PUPPETCONFIG $PUPPETCONFIG.bak-landscape
fi

# Write the new puppet config file
cat > $PUPPETCONFIG << END
# Default puppet configuration pushed by Landscape.
[agent]
  server = puppet.example.com
  report = true
  pluginsync = true
END

# Use puppet itself to make sure the puppet service starts on boot
puppet resource service puppet ensure=running enable=true


To build the package, change to your working directory that contains the “landscape-puppet” directory then execute the following command:

1
$ dpkg-deb -b landscape-puppet landscape-puppet_0.1_all.deb


If all goes well you should now have a landscape-puppet.deb file in the current directory. Congratulations, you’ve built a meta-package!

This is just a touch on meta-packages. For more info check out https://help.ubuntu.com/community/MetaPackages

Landscape Repository Management
Now that we have a package, we’re going to use Landscape’s new Repository Management features to create a new apt repository and upload our package. Because this feature is so new, it’s only available through the API for now. I’ll walk you through setting things up.

Landscape’s Repository Management supports a few different use cases:

Landscape can mirror public repositories – like the official Ubuntu archive – into your local network. This gives you LAN speeds for updates and lets YOU control when packages get deployed.

Landscape can stage repositories. You might want to try new packages on a staging system before you roll them out to production. Landscape lets you selectively pull packages into your production repository after you’ve tested them on your staging system – all managed by Landscape.

And for this example, we’re using the Upload Repository. This lets you create an empty repository and upload your own packages for distribution.

In all cases Landscape takes care of the details for you. GPG keys are verified when mirroring external repositories, and keys are distributed to clients automatically when creating your own repositories. Everything is cryptographically verified.

To use the API, you first have to get your API credentials. Follow the steps on https://help.landscape.canonical.com/LDS/BetaReleaseNotes under the “API Getting Started” section. Once you’ve got your credentials set up in the appropriate environment variables use the following commands to set up the repository:

Create a new GPG key to sign your repository:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
$ cat > gpg-batch-commands << EOF
Key-Type: DSA
Key-Length: 2048
Subkey-Type: ELG-E
Subkey-Length: 2048
Name-Real: Landscape Repository Signing Key
Name-Comment: Landscape
Name-Email: landscape@example.com
Expire-Date: 0
%pubring landscape.pub
%secring landscape.sec
%commit
EOF

$ gpg --batch --gen-key gpg-batch-commands

$ gpg --no-default-keyring --secret-keyring ./landscape.sec \
  --keyring ./landscape.pub --export-secret-keys \
  -a > landscape-signing-key-secret


Now import your new secret key. Landscape will use it to sign your new repository:

1
2
3
4
5
$ landscape-api import-gpg-key upload-signing-key landscape-signing-key-secret
{u'fingerprint': u'c7da:4521:a948:600a:bd26:6e69:2fec:1d65:524a:3a15',
 u'has_secret': True,
 u'id': 3,
 u'name': u'upload-signing-key'}

 

Now that we have a key, we can configure our repository:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
$ landscape-api create-distribution landscape
{u'access_group': u'global',
 u'creation_time': u'2012-09-18T15:21:36Z',
 u'name': u'landscape',
 u'series': []}

$ landscape-api create-series upload landscape
{u'creation_time': u'2012-09-18T15:23:21Z', u'name': u'upload', u'pockets': []}

$ landscape-api create-pocket ubuntu upload landscape main amd64 upload upload-signing-key --upload-allow-unsigned
{u'apt_source_line': u'deb http://localhost:8080/repository/onward/landscape upload-ubuntu main',
 u'architectures': [u'amd64'],
 u'components': [u'main'],
 u'creation_time': u'2012-09-18T15:28:54Z',
 u'gpg_key': {u'fingerprint': u'c7da:4521:a948:600a:bd26:6e69:2fec:1d65:524a:3a15',
 u'has_secret': True,
 u'id': 3,
 u'name': u'upload-signing-key'},
 u'mode': u'upload',
 u'name': u'ubuntu',
 u'upload_allow_unsigned': True,
 u'upload_gpg_keys': []}

 

Now that we have our repository set up we need to upload our deb. To send packages to our upload pocket, we need to generate a “.changes” file which describes our upload. The “.changes” file contains additional information to help Landscape get your package into the right repository. Just use these commands to generate the “.changes” file and upload both files to Landscape.

If you are familiar with the “dput” command it can be used instead of the curl commands below. It has many options for checking your debs before upload. For this example however the curl commands will work fine.

1
2
3
4
5
$ changestool --create my.changes adddeb landscape_puppet_0.1_all.deb
$ changestool --create my.changes setdistribution upload-ubuntu

$ curl --upload-file landscape-puppet_0.1_all.deb http://lds.example.com/repository/standalone/landscape/upload/ubuntu/
$ curl --upload-file my.changes http://lds.example.com/repository/standalone/landscape/upload/ubuntu/


Now that our package is all set up in its repository, we will create a repository profile so we can make sure this repository is available to any machine with the “puppet” tag.

WARNING: Once you associate a repository profile to a computer in Landscape, it takes over management of your sources.list and sources.list.d on that computer. This means if you still want to use the official Ubuntu archive you need to make a repository profile for it. Take a look at the create-apt-source command and the documentation here for more details.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
$ landscape-api create-repository-profile landscape-upload
{u'access_group': u'global',
 u'all_computers': False,
 u'apt_sources': [],
 u'description': u'',
 u'id': 4,
 u'name': u'landscape-upload',
 u'pockets': [],
 u'tags': []}

$ landscape-api add-pockets-to-repository-profile landscape-upload ubuntu upload landscape
{u'access_group': u'global',
 u'all_computers': False,
 u'apt_sources': [],
 u'description': u'',
 u'id': 4,
 u'name': u'landscape-upload',
 u'pockets': [{u'apt_source_line': u'deb http://localhost:8080/repository/onward/landscape upload-ubuntu main',
 u'architectures': [u'amd64'],
 u'components': [u'main'],
 u'creation_time': u'2012-09-18T15:28:54Z',
 u'gpg_key': {u'fingerprint': u'c7da:4521:a948:600a:bd26:6e69:2fec:1d65:524a:3a15',
 u'has_secret': True,
 u'id': 3,
 u'name': u'upload-signing-key'},
 u'mode': u'upload',
 u'name': u'ubuntu',
 u'upload_allow_unsigned': True,
 u'upload_gpg_keys': []}],
 u'tags': []}

$ landscape-api associate-repository-profile landscape-upload --tags puppet
{u'access_group': u'global',
 u'all_computers': False,
 u'apt_sources': [],
 u'description': u'',
 u'id': 4,
 u'name': u'landscape-upload',
 u'pockets': [{u'apt_source_line': u'deb http://localhost:8080/repository/onward/landscape upload-ubuntu main',
 u'architectures': [u'amd64'],
 u'components': [u'main'],
 u'creation_time': u'2012-09-18T15:28:54Z',
 u'gpg_key': {u'fingerprint': u'c7da:4521:a948:600a:bd26:6e69:2fec:1d65:524a:3a15',
 u'has_secret': True,
 u'id': 3,
 u'name': u'upload-signing-key'},
 u'mode': u'upload',
 u'name': u'ubuntu',
 u'upload_allow_unsigned': True,
 u'upload_gpg_keys': []}],
 u'tags': [u'puppet']}

 

Landscape Package Profiles
So now we have our deb package sitting in a new apt repository ready to go. The final step is to create a package profile so the new package gets installed. We’ll do this using the GUI.



First, choose “Package Profiles” from the main menu:



Now, select “Add Package Profile”:



Fill in the form with your details:



Associate your new package profile with the “puppet” tag:



Finally you can see the status of the profile and how many computers have the profile installed:

Pulling it all together
Now that everything is set up, we really see the power of this technique. Pick any computer in Landscape and add the “puppet” tag. This will automatically add our new apt repository to the computers apt sources, then it will automatically install our new meta-package. The new meta-package will automatically install puppet and all its dependencies, then the postinst script will finish the whole thing off by registering the puppet agent to the puppet-master.

By combining Landscape features we can apply some very powerful automation to our infrastructure. In this case, once puppet is registered the puppet-master will finish off customising the system according to our system catalogues.

In a later post I’ll show how to use Landscape as an External Node Classifier for Puppet for even more automation!

Mike Milner

Landscape with Nagios

Nagios is not the prettiest monitoring tool, but I do love it. There is something wonderful about all those little green squares telling me that everything is up and working.

I still remember the first time I set up a small Nagios system in my organization – I wrote every configuration file by hand and watched the green squares multiply as I added more hosts and services. It was fun at first, but as the system grew, editing all those configuration files by hand quickly became tedious. If you’re like me, the need to automate tedious tasks is overwhelming. That’s where the Landscape API comes in to make this simple.

At it’s heart, Landscape is a tool to help you manage your computing architecture. Landscape allows you to assign tags to every computer in your infrastructure. We’re going to use the Landscape API plus these tags to discover new computers and automatically configure them in Nagios. Add a computer to Landscape and it will automatically be added to Nagios with the appropriate hostgroup and services automatically configured.

The Landscape API
To use the Landscape API you need three pieces of information: the Landscape API endpoint URI, the user key, and the secret for your user.

To find your API credentials, log in to the Landscape web interface and click your username on the top right. Your API credentials are in the “API Access” section.

For this example, we’ll use the python API client provided with Landscape (NOTE, you must install the landscape-api package first). Here’s how to query for a list of all the computers in registered with Landscape:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
from landscape_api.base import API

landscape_uri = "https://landscape.canonical.com/api/"
landscape_key = "43NW6OV71L32CSOPCJGX"
landscape_secret = "agBf3v267DqO8vtVRnzjseWfYdV4ueklj5a81iIT"
api = API(landscape_uri, landscape_key, landscape_secret)

api.get_computers()
[{u'access_group': u'server',
u'comment': u'',
u'hostname': u'appserv1',
u'id': 1,
u'last_exchange_time': None,
u'last_ping_time': u'2012-09-07T15:19:22Z',
u'reboot_required_flag': False,
u'tags': [u'lucid', u'server', u'webfarm'],
u'title': u'Application Server 1',
u'total_memory': 1024,
u'total_swap': 1024},
{u'access_group': u'server',
u'comment': u'',
u'hostname': u'appserv2',
u'id': 2,
u'last_exchange_time': None,
u'last_ping_time': u'2012-09-07T15:18:22Z',
u'reboot_required_flag': False,
u'tags': [u'nagios', u'precise', u'server', u'webserver'],
u'title': u'Application Server 2',
u'total_memory': 1024,
u'total_swap': 1024}]

 

For this example we will also apply a filter to the get_computers() call so we only get back computers tagged with “nagios” and we will ask for the networking information to be included in the response.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
api.get_computers(query="tag:nagios", with_networking=True)
[{u'access_group': u'server',
u'comment': u'',
u'hostname': u'appserv2',
u'id': 2,
u'last_exchange_time': None,
u'last_ping_time': u'2012-09-07T15:18:22Z',
u'network_devices': [{u'broadcast_address': u'192.168.0.255',
u'interface': u'eth0',
u'ip_address': u'192.168.0.61',
u'mac_address': u'00:1f:d9:ce:e1:69',
u'netmask': u'255.255.255.0'}],
u'reboot_required_flag': False,
u'tags': [u'nagios', u'precise', u'server', u'webserver'],
u'title': u'Application Server 2',
u'total_memory': 1024,
u'total_swap': 1024}]

 

Nagios Configuration
Once we get the computers we’re interested in, we need to add them to the Nagios config. This example creates a separate configuration file for each Landscape computer. These should all be stored in a directory which Nagios will read.

To tell Nagios to process all files in a directory (and it’s subdirectories) you use the cfg_dir= command like this:

1
cfg_dir=/path/to/host/files

 

Now all we have to do is create files in that directory for each host we get back from Landscape.

A Nagios host configuration file looks something like this:

1
2
3
4
5
6
7
define host{
    use          linux-host
    host_name    appserv2
    alias        Application Server 2
    address      192.168.0.61
    hostgroups   server, webserver
}

 

This file defines the host by specifying the host_name, alias, and address. The “use” parameter causes this host to inherit all the settings of the “linux-host” template. The “hostgroups” parameter puts this host under the “server” hostgroup and the “webserver” hostgroup.

The hostgroups define all the services that need to be monitored for any host that is part of the hostgroup. In this example, our host will get all services from the “server” hostgroup and the “webserver” hostgroup.

The Script
Now that we have the basics of talking to Landscape and writing config files for Nagios, we’re going to pull it all together.


The tags we get from landscape will be used to link the host to Nagios’s hostgroups. The script is designed to be run repeatedly to keep the host definitions up to date. In my setup I run it in a cronjob every five minutes – new hosts and tag changes are picked up fairly quickly.

The script is fairly simple. I’ll walk through it piece by piece. For the full listing with full comments and doc string see the end of the articale.

First, we import the libraries we need and define our configuration values:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# List of valid hostgroups - Landscape tags not in this list are ignored.
# You must have a Nagios hostgroup defined for each entry in the list.
VALID_HOSTGROUPS = ["server", "webfarm"]

# A host template name to use for all the generated hosts. This host template
# must already be defined in your Nagios configuration.
HOST_TEMPLATE = "ubuntu-host"

# The directory to store the generated host files. It must be loaded into
# the Nagios config using a cfg_dir= command.
NAGIOS_HOST_DIRECTORY = "nagios"

 

Now we ask landscape for a list of all computers with the “nagios” tag set and update each computer:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
api = API(LANDSCAPE_URI, LANDSCAPE_KEY, LANDSCAPE_SECRET)

try:
    # Get all computers tagged with "nagios"
    computers = api.get_computers(query="tag:nagios", with_network=True)
except errors.InvalidQueryError:
    # If there has never been a computer tagged "nagios" we get a
    # query error
    computers = []

# Update the nagios config for each computer
for computer in computers:
    update_computer(computer)

 

For each computer we extract the information Nagios needs. In this example we use the first IP address available for the host. If your network is more complex you may have to add some more smarts here to use the IP address reachable by Nagios. Once the config is generated we write it to a file in the Nagios hosts directory:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
def update_computer(computer):
    """Generates a host config file for the computer and saves it to disk."""

    hostname = computer["hostname"]
    title = computer["title"]
    tags = computer["tags"]
    ips = [net["ip_address"] for net in computer["network_devices"]]

    # Filter out tags that aren't valid hostgroups
    hostgroups = set(VALID_HOSTGROUPS).intersection(set(tags))

    # If there is more than one IP address, use the first
    ip = ips[0]

    # Use the hostname as the nagios filename
    filename = "%s.cfg" % hostname
    filepath = os.path.join(NAGIOS_HOST_DIRECTORY, filename)

    # Generate the config file contents
    config = generate_nagios_config(HOST_TEMPLATE,
                                    hostname, title, ip, hostgroups)

    # Write out the config file
    with open(filepath, "wb") as f:
        f.write(config)

 

Here’s how the nagios host configuration file is generated:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
def generate_nagios_config(template, hostname, alias, ip, hostgroups):
    """Generates the contents of a Nagios host config file."""
    lines = [
        "# Automatically generated from Landscape API data.",
        "# Any changes will be overwritten.",
        "define host{",
        " use %s" % template,
        " host_name %s" % hostname,
        " alias %s" % alias,
        " address %s" % ip,
        " hostgroups %s" % (", ".join(hostgroups)),
        "}"]
    return "\n".join(lines)

 

Finally, we ask nagios to reload the configuration files to see the new data.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
def reload_config():
    """
 Use Ubuntu init script to reload the Nagios configuration. Note that
 the configuration is checked before reloading to prevent errors from
 stopping the nagios process.
 """
    try:
        subprocess.check_output(["service", "nagios3", "reload"],
                                stderr=subprocess.STDOUT)
    except subprocess.CalledProcessError as e:
        sys.stderr.write("FAILED RELOADING NAGIOS CONFIG\n")
        sys.stderr.write(e.output)
        sys.exit(1)

 

When the script is run, the new computer shows up immediately in Nagios:

And within a few minutes everything looks OK!

This simple example shows how you can integrate Landscape within your existing infrastructure. The Landscape API allows you to reach all your Ubuntu infrastructure using simple API commands from python or the command line.

We’ve barely scratched the surface of what the API can do. Check out the API Documentation to learn more.

The complete script is available here: http://paste.ubuntu.com/1200590/

Federico Lucifredi

The Landscape Team is happy to announce the release of an update to Landscape Dedicated Server, the version of our Ubuntu Systems Management service deployed behind the firewall at customer sites.

LDS 11.07.1 (11.07.20120217-2) aligns the Dedicated Server with updates recently introduced on http://landscape.canonical.com.  It includes support for the new AWS region in São Paulo (sa-east-1) and Ubuntu Oneiric (11.10) images, as well as a number of security fixes detailed in the release notes.

The most noteworthy new feature is support for OpenStack as a custom EC2 compatible endpoint.  I was able to set up CanoniStack (our internal R&D Cloud) as a cloud deployment target in my Landscape account and start launching instances from Landscape quicker than it is taking me to write about it – and it is all fronted in our standard cross-cloud interface in Landscape.

 

Landscape + OpenStack

Landscape launching two Oneiric instances on an OpenStack Cloud.

 

Stefan Stasik

We’re releasing a new version of Landscape on September 12th. While we roll-out the code to Landscape’s servers, we’ll have to take the service offline.

Goes offline: 15:00 September 12th 2011 UTC
Expected back online: 15:30 September 12th 2011 UTC

Keep an eye on this blog after the release to find out what’s new!

We’re sorry for the downtime you’ll experience during the roll-out.

Tom Haddon

We’re releasing a new version of Landscape on June 20th. While we roll-out the code to Landscape’s servers, we’ll have to take the service offline.

Goes offline: 15:00 June 20th 2011 UTC
Expected back online: 15:30 June 20th 2011 UTC

Keep an eye on this blog after the release to find out what’s new!

We’re sorry for the downtime you’ll experience during the roll-out.