A lot of activity goes on within a cloud environment. It is a mix of hardware, operating systems, virtual machine managers, the OpenStack services, cloud-user activity such as creating instances and attaching storage, the network underlying the whole, and finally end-users using the applications running on the various instances.
The generation and collection of logs is an important component of securely monitoring an OpenStack infrastructure. Logs provide visibility into the day-to-day actions of administrators, tenants, and guests, in addition to the activity in the compute, networking, and storage and other components that comprise your OpenStack deployment.
The basics of logging: configuration, setting log level, location of the log files, and how to use and customize logs, as well as how to do centralized collections of logs is well covered in the OpenStack Operations Guide.
Logs are not only valuable for proactive security and continuous compliance activities, but they are also a valuable information source for investigating and responding to incidents.
For instance, analyzing the access logs of Identity Service or its replacement authentication system would alert us to failed logins, their frequency, origin IP, whether the events are restricted to select accounts etc. Log analysis supports detection.
On detection, further action may be to black list an IP, or recommend strengthening user passwords, or even de-activating a user account if it is deemed dormant.
Monitoring events is more pro-active and provides real-time detection and response. There are several tools to aid in monitoring.
In the case of an OpenStack cloud instance, we need to monitor the hardware, the OpenStack services, and the cloud resource usage. The last stems from wanting to be elastic, to scale to the dynamic needs of the users.
Here are a few important use cases to consider when implementing log aggregation, analysis and monitoring. These use cases can be implemented and monitored through various commercial and open source tools, homegrown scripts, etc. These tools and scripts can generate events that can then be sent to the administrators through email or integrated dashboard. It is important to consider additional use cases that may apply to your specific network and what you may consider anomalous behavior.
Detecting the absence of log generation is an event of high value. Such an event would indicate a service failure or even an intruder who has temporarily switched off logging or modified the log level to hide their tracks.
Application events such as start and/or stop that were unscheduled would also be events to monitor and examine for possible security implications.
OS events on the OpenStack service machines such as user logins, restarts also provide valuable insight into use/misuse
Being able to detect the load on the OpenStack servers also enables responding by way of introducing additional servers for load balancing to ensure high availability.
Other events that are actionable are networking bridges going down, ip tables being flushed on compute nodes and consequential loss of access to instances resulting in unhappy customers.
To reduce security risks from orphan instances on a user/tenant/domain deletion in the Identity service there is discussion to generate notifications in the system and have OpenStack components respond to these events as appropriate such as terminating instances, disconnecting attached volumes, reclaiming CPU and storage resources etc.
A cloud will host many virtual instances, and monitoring these instances goes beyond hardware monitoring and log files which may just contain CRUD events.
Security monitoring controls such as intrusion detection software, antivirus software, and spyware detection and removal utilities can generate logs that show when and how an attack or intrusion took place. Deploying these tools on the cloud machines provides value and protection. Cloud users, those running instances on the cloud may also want to run such tools on their instances.
