The NeCTAR Dashboard is the simplest way to manage your instances in projects. However, there is also an Application Programming Interface (API) which makes additional functionality available to advanced users. The NeCTAR Research Cloud is managed with OpenStack, and users also have access to the same OpenStack API as is used internally.
The API has a few different methods by which you can access it:
- Command line clients.
- OpenStack Python APIs.
- Direct HTTPS requests.
The most common way is to use one or more command line clients. These are scripts pre-written in Python and ready to be run by users. Most often, you will want to use the general-purpose openstack-client. Openstack-client is phasing out a variety of more specialised legacy clients such as nova and glance. The distinction between the openstack-client and the others will be explored in more detail below.
Secondly, there is a Python API for OpenStack. This is useful to Python programmers who need to create custom scripts.
Finally, it is possible to access the API using traditional Unix command line tools such as curl. This can be useful because not all aspects of the API are available using other methods. Furthermore, an advanced user might be able to perform tasks faster using a custom script using curl instead of relying on the other methods. It is fairly rare to want to use this method even amongst people who use the API all the time, but it can be useful to know the option is available. Note that we do not usually provide support for this kind of API access.
The first two types of API access have the same installation process. If you install one, you will have them both.
Both methods require installing Python packages. Again, there are a variety of ways to do this. We will focus on the method that is currently preferred by most Python users: pip + virtualenv.
Pip is a tool for fetching and managing Python packages from an online repository. A virtualenv is a way to isolate Python packages so they do not interfere with the rest of your system. Both of these tools are free, open source and cross-platform, so there will be fairly similar procedures regardless of whether you use Windows, Mac OS X, Linux, or some other operating system.
We will now assume that after following those instructions, you have created and activated a virtualenv. The next step is to use Pip to install the OpenStack API software into the virtualenv:
pip install python-openstackclient
Once you succeed with this step you are ready to begin accessing the API.
Using the API requires authenticating with the NeCTAR Research Cloud. Four variables must be set in your shell environment for authentication to take place:
- OS_AUTH_URL: Where should the authentication request be sent?
- OS_PROJECT_NAME and/or OS_PROJECT_ID: "Tenant" is an older OpenStack term that is synonymous with "project". Only one of these two variables needs to be set.
- OS_USERNAME: Your institutional email address.
- OS_PASSWORD: A special password used only for API access.
These variables are usually set from a script that you run in each fresh shell environment before accessing the API. The below instructions walk you through how to obtain a default script.
Log in to the NeCTAR Research Cloud Dashboard.
Ensure you are in the correct project using the drop down list beside the NeCTAR logo at the top-left of the page.
Click 'Access & Security' then go to the 'API Access' tab.
Click the 'Download OpenStack RC File' button. Save the file into a directory that you can easily find again in future.
You will also need a special password for API access:
At the top right of the Dashboard, click your email address to drop down a menu; from that menu proceed to Settings.
Click 'Reset Password'. A password will appear on the screen. Save this password somewhere secure. Anyone with this password will have complete control over your NeCTAR project.
Important: clicking 'Reset Password' in this way does not change your institutional or Dashboard login password! It only affects the password used for API authentication. This is held entirely separate to everything else.
Authentication for Command Line API Clients
Assuming you have a Unix-like command line, you can make the authentication variables available in your current environment by sourcing the script you downloaded from the Dashboard. For example:
At this point you will be asked for the special API password you obtained from the Dashboard earlier.
If you understand Unix shell script, it is usually safe to make reasonable adjustments to the authentication script. For example, you might wish to hardcode the password into the script so you are not asked for it each time. However, we do not provide support for this kind of customisation and you then furthermore need to take extra care with the security of your script.
Authentication for python swift API
The sample Python code further down includes an example of authentication.
Command Line API Client Usage
To get information useful for managing instances in your project:
|openstack help or openstack --help||Discover the available options and sub-commands|
|openstack help <command-name>||Information on a specific sub-command|
|openstack server list||To list all active instances in one project|
|openstack flavor list||To list all available flavors|
|openstack image list||To list all available images|
|openstack security group list||To list all available security groups|
|openstack keypair list||To list all available key pairs|
|openstack availability zone list||To list all available availability zones|
- You might find some options unavailable due to NeCTAR security policies.
- In practice, some of the legacy clients like nova and glance might prove to be faster or have more features available. A useful comparison between openstack-client and the others is available at this rosetta stone. Sometimes differences can only be found by experimenting or talking to more experienced people.
Managing an Instance
To boot a new instance, you need several items of information to pass as arguments to the client:
- Image ID
- Flavor name or ID
- Keypair name
- Security group names or ID
You can use the above mentioned commands to acquire this information.
It is assumed that you have already created some security groups via the Dashboard. It is also possible to manage security groups using the API, but that is outside the scope of this document.
Once you have all that information available, the following command will launch a new instance:
openstack server create --flavor <flavor-name> --image <image-id> \ --key-name <keypair> --security-group <sec-gp-1> \ --security-group <sec-gp-2> <server-name>
You can also specify option --availability-zone to launch an instance in a designated zone and --user-data for an initialisation script to run during instance creation.
You can see this website for some useful scripts you might wish to use.
|Commands for instance management||Action|
|openstack server||To boot a stopped instance|
|openstack server||To stop an instance|
|openstack server||To suspend an instance|
|openstack server||To resume an instance|
|openstack server||To show details of an instance|
|openstack server||To create a snapshot of a instance|
The openstack command has extensive built-in help. You can also find more information about openstack-client from OpenStack.
The Python APIs for OpenStack come in two flavors.
- There are separate Python APIs that correspond to the different components of OpenStack. These are provided by the "python-<service>client" pip packages, and are the primary APIs maintained by the respective component projects.
- There is a also combined Python API called the OpenStackSDK.
First, we will present some Python code that that uses the Python API for Nova to launch an instance. As with all OpenStack applications it also uses the Python Keystone API to authenticate and obtain the session that is needed for the Nova client. To run the code, you need to use pip to install the "python-novaclient" and "python-keystoneclient" packages.
The following Python script will launch a new instance:
# Config our auth settings from keystoneauth1 import loading loader = loading.get_plugin_loader('password') authurl = "https://keystone.rc.nectar.org.au:5000/v3/" user = "firstname.lastname@example.org" passwd = "XXXXXXX" proj = "<project-id>" auth = loader.load_from_options( auth_url=authurl, username=user, password=passwd, project_id=proj, user_domain_name="Default") # Use those settings to create a session. from keystoneauth1 import session sess = session.Session(auth=auth) # Create a novaclient in our session. from novaclient import client nova = client.Client("2", session = sess) # Use the novaclient to launch a server server_name = 'xenomorph' image = '0f72065c-2937-4ece-b3d5-424808ca7cdf' # Image ID for Ubuntu 16.04 flavor = '0' # Flavor ID for "m1.small" security_groups = ['ssh', 'icmp/ping'] key = 'keypair-name' nova.servers.create( server_name, image, flavor, security_groups=security_groups, key_name=key)
You can use the OpenStack CLI tool to lookup more up-to-date flavor and image ids for the above example. You can find more information on the Nova Python API, the Keystone Python API, and other client APIs from the OpenStack documentation.
We won't go into details of how to use the OpenStackSDK except to provide a simple example of how the authentication parameters need to be provided for Nectar:
import openstack # openstack.enable_logging(debug=True) # Establish connection conn = openstack.connection.Connection( region_name='Melbourne', auth=dict( auth_url='https://keystone.rc.nectar.org.au:5000/v3/', project_name='my_project', email@example.com', password='mypassword', user_domain_name='Default', project_domain_name='Default') ) # Do something conn.compute.servers()
Direct HTTPS Access with Curl
Some examples available from OpenStack might help get you started if you need to access the API using curl.