Nectar Research Cloud - 2019 User Survey Report Summary and Recommendations

ARDC undertook its Nectar Research Cloud user survey during November 2019. As per previous years the survey was based on using the Net Promoter Score system but this year, we asked additional questions pertaining to some of the Nectar Research Cloud services and also training. The survey was constructed and managed using an online survey tool from eSurv.org. Invitations were sent to 3,550 users of the Nectar Research Cloud, with 496 participating in the survey (14% response rate). There was a total of 50 (10%) detractors , 124 (25%) passives, and 322 (65%) promoters, resulting in a Net Promoter Score (NPS) of 55, on a possible scale of -100 to +100. This indicates that the Nectar Research Cloud is viewed positively with an excellent level of satisfaction amongst responders and sees a much-improved score this year compared to the score of 24 from the last NPS score in 2017. Much more positive feedback was received than negative. This year the survey was not sent to all users that have ever created a Nectar Cloud account as per previous surveys but instead sent to active users of the Nectar Research Cloud. Active users are defined as users that are attached to an allocation that has launched an instance in the last 12 months.

There were some common themes that emerged from the survey responses for the Nectar Research Cloud to improve upon that were similar to previous surveys. These  were:

1. Usability is again this year the largest factor that underpinned responses. Positive comments far outweighed detractor comments. Common detractor comments relating to usability were about technical  vocabulary being difficult to understand, and the amount of time and technical skills required to get started were limiting factors. The  interface was also mentioned as being complicated and could be improved.
2. Reliability was a concern. Detractor comments about reliability focused on too many outages and scheduled down times for upgrades and maintenance. 
3. Detractor comments pertaining to performance and capacity focused on service slowness and lack of availability of resources in the cloud.

Based on the survey responses, it is recommended that ARDC consider:

1. Continue a program of user surveys utilising NPS to track the  Nectar Research Cloud’s NPS score over time and to collect valuable feedback to drive continuous improvement, enhance services and develop new functionality. Additional questions that focus on specific aspects of services to inform service performance, improvement and development are useful to continue to include but descriptions of services should be included in the survey question when questions are asking survey participants to answer about a particular service (not in a link from the question as survey participants are unlikely to click on the links). 

2. Continue to implement improvements to improve the ease of use of services and develop mechanisms to minimise the technical barrier to entry.

3. Review when scheduled down times for upgrades and maintenance occur and work with Nodes to better manage planned outages to reduce impact on users. 

4. Consider and evaluate enabling the following capabilities requested from respondents:

   • Larger RAM quotas

   • Automated Snapshot based backups

   • Support for Windows

   • Simpler set up and user guidelines and more prominent access to self help/guidelines

   • GPU computing

   • Work with AARNet, Nodes, HPC centres, and universities to enable faster transfer for moving data between HPC’s, VMs and external storage across Australia

   • Better integration with Cloudstor

5. Consider developing application packages for Anaconda, Matlab and Gitlab or Git repository hosting applications as there were multiple requests from respondents for these application packages to the question of “Are there any broadly used applications you would like to see published in the Application Catalogue?”. There were also multiple requests for RStudio Server and Juypter which are already available in the application catalogue. So better promotion to users about the applications available in the application catalogue is recommended.

6. Provide more resources to improve user support, training materials and provision of  training to help make the Nectar Research Cloud easier for users to use by assisting them in acquiring the skills and knowledge to optimise their usage of the Nectar Research Cloud. There were lots of suggestions received for additional training materials and courses so more resources should be provided to meet this demand as the currently funded resources are not sufficient.  This demand is not a new discovery and the previously drafted Training Strategy Proposal outlines and recommends a national training model to support this demand. We should also consider including linkages to external training resources already freely available to meet users training requirements in the Nectar Research Cloud online training and knowledge base which will help users develop general technical skills and knowledge to effectively utilise the Nectar Research Cloud  (e.g. external Unix scripting training resources).