Manage your instance via API Command Line

The NeCTAR Dashboard is the simplest way to manage your instances in projects. However, there is also an Application Programming Interface (API) which makes additional functionality available to advanced users. The NeCTAR Research Cloud is managed with OpenStack, and users also have access to the same OpenStack API as is used internally.

The API has a few different methods by which you can access it:

1. Command line clients.
2. OpenStack Python APIs.
3. Direct HTTPS requests.

The most common way is to use one or more command line clients. These are scripts pre-written in Python and ready to be run by users. Most often, you will want to use the general-purpose openstack-client. Openstack-client is phasing out a variety of more specialised legacy clients such as nova and glance. The distinction between the openstack-client and the others will be explored in more detail below.

Secondly, there is a Python API for OpenStack. This is useful to Python programmers who need to create custom scripts.

Finally, it is possible to access the API using traditional Unix command line tools such as curl. This can be useful because not all aspects of the API are available using other methods. Furthermore, an advanced user might be able to perform tasks faster using a custom script using curl instead of relying on the other methods. It is fairly rare to want to use this method even amongst people who use the API all the time, but it can be useful to know the option is available. Note that we do not usually provide support for this kind of API access.

Installation

The first two types of API access have the same installation process. If you install one, you will have them both.

Both methods require installing Python packages. Again, there are a variety of ways to do this. We will focus on the method that is currently preferred by most Python users: pip + virtualenv.

Pip is a tool for fetching and managing Python packages from an online repository. A virtualenv is a way to isolate Python packages so they do not interfere with the rest of your system. Both of these tools are free, open source and cross-platform, so there will be fairly similar procedures regardless of whether you use Windows, Mac OS X, Linux, or some other operating system.

Follow the instructions to install python-pip relevant to your operating system, then follow the virtualenv instructions.

We will now assume that after following those instructions, you have created and activated a virtualenv. The next step is to use Pip to install the OpenStack API software into the virtualenv:

pip install python-openstackclient

Once you succeed with this step you are ready to begin accessing the API.

Configuration

Using the API requires authenticating with the NeCTAR Research Cloud. Four variables must be set in your shell environment for authentication to take place:

• OS_AUTH_URL: Where should the authentication request be sent? 
• OS_PROJECT_NAME and/or OS_PROJECT_ID: "Tenant" is an older OpenStack term that is synonymous with "project". Only one of these two variables needs to be set.
• OS_USERNAME: Your institutional email address.
• OS_PASSWORD: A special password used only for API access.

These variables are usually set from a script that you run in each fresh shell environment before accessing the API. The below instructions walk you through how to obtain a default script.

• Log in to the NeCTAR Research Cloud Dashboard.

• Ensure you are in the correct project using the drop down list beside the NeCTAR logo at the top-left of the page.

• Click 'Access & Security' then go to the 'API Access' tab.

• Click the 'Download OpenStack RC File' button. Save the file into a directory that you can easily find again in future.

You will also need a special password for API access:

• At the top right of the Dashboard, click your email address to drop down a menu; from that menu proceed to Settings.

• Click 'Reset Password'. A password will appear on the screen. Save this password somewhere secure. Anyone with this password will have complete control over your NeCTAR project.

Important: clicking 'Reset Password' in this way does not change your institutional or Dashboard login password! It only affects the password used for API authentication. This is held entirely separate to everything else.

Authentication for Command Line API Clients

Assuming you have a Unix-like command line, you can make the authentication variables available in your current environment by sourcing the script you downloaded from the Dashboard. For example:

source /path/to/auth/script.sh

At this point you will be asked for the special API password you obtained from the Dashboard earlier.

If you understand Unix shell script, it is usually safe to make reasonable adjustments to the authentication script. For example, you might wish to hardcode the password into the script so you are not asked for it each time. However, we do not provide support for this kind of customisation and you then furthermore need to take extra care with the security of your script.

Authentication for python swift API

The sample Python code further down includes an example of authentication.

Command Line API Client Usage

To get information useful for managing instances in your project:

Some notes:

• You might find some options unavailable due to NeCTAR security policies.
• In practice, some of the legacy clients like nova and glance might prove to be faster or have more features available. A useful comparison between openstack-client and the others is available at this rosetta stone. Sometimes differences can only be found by experimenting or talking to more experienced people.

Managing an Instance

To boot a new instance, you need several items of information to pass as arguments to the client:

• Image ID
• Flavor name or ID
• Keypair name     
• Security group names or ID

You can use the above mentioned commands to acquire this information.

It is assumed that you have already created some security groups via the Dashboard. It is also possible to manage security groups using the API, but that is outside the scope of this document.

Once you have all that information available, the following command will launch a new instance:

openstack server create --flavor <flavor-name> --image <image-id> \
    --key-name <keypair> --security-group <sec-gp-1> \
    --security-group <sec-gp-2> <server-name>

You can also specify option --availability-zone to launch an instance in a designated zone and --user-data for an initialisation script to run during instance creation.

You can see this website for some useful scripts you might wish to use.

The openstack command has extensive built-in help.  You can also find more information about openstack-client from OpenStack.

Python APIs

The Python APIs for OpenStack come in two flavors.

• There are separate Python APIs that correspond to the  different components of OpenStack. These are provided by the "python-<service>client" pip packages, and are the primary APIs maintained by the respective component projects. 
• There is a also combined Python API called the OpenStackSDK. 

First, we will present some Python code that that uses the Python API for Nova to launch an instance.  As with all OpenStack applications it also uses the Python Keystone API to authenticate and obtain the session that is needed for the Nova client. To run the code, you need to use pip to install the "python-novaclient" and "python-keystoneclient" packages.

The following Python script will launch a new instance:

# Config our auth settings
from keystoneauth1 import loading
loader = loading.get_plugin_loader('password')
authurl = "https://keystone.rc.nectar.org.au:5000/v3/"
user = "me@my.edu"
passwd = "XXXXXXX"
proj = "<project-id>"
auth = loader.load_from_options(
        auth_url=authurl, username=user, 
        password=passwd, project_id=proj,
        user_domain_name="Default")

# Use those settings to create a session.
from keystoneauth1 import session
sess = session.Session(auth=auth)

# Create a novaclient in our session.
from novaclient import client
nova = client.Client("2", session = sess)

# Use the novaclient to launch a server
server_name = 'xenomorph'
image = '0f72065c-2937-4ece-b3d5-424808ca7cdf' # Image ID for Ubuntu 16.04
flavor = '0' # Flavor ID for "m1.small"
security_groups = ['ssh', 'icmp/ping']
key = 'keypair-name'
nova.servers.create(
        server_name, image, flavor, 
        security_groups=security_groups, 
        key_name=key)

You can use the OpenStack CLI tool to lookup more up-to-date flavor and image ids for the above example. You can find more information on the Nova Python API, the Keystone Python API, and other client APIs from the OpenStack documentation.

We won't go into details of how to use the OpenStackSDK except to provide a simple example of how the authentication parameters need to be provided for Nectar:

import openstack

# openstack.enable_logging(debug=True)

# Establish connection
conn = openstack.connection.Connection(
    region_name='Melbourne',
    auth=dict(
        auth_url='https://keystone.rc.nectar.org.au:5000/v3/',
        project_name='my_project',
        username='me@my.edu',
        password='mypassword',
        user_domain_name='Default',
        project_domain_name='Default')
    )

# Do something
conn.compute.servers()

Direct HTTPS Access with Curl

Some examples available from OpenStack might help get you started if you need to access the API using curl.