Async Log4j in IoT Environment |

Security of IoT environment is the need of time to identify unauthorized access and reconfiguration of IoT devices. IoT networks required logging to analyze the data flow and allow different tools to do forensic analysis of logs in-case of security breaches.

We are now living in a world of IoT devices. Many IoT Platforms are developed in current years to support different IoT devices. Each IoT platform needs a logging mechanism to inform the user about the state of the system in case of high availability.
We have recently moved one of the customer’s ( IoT platform) sync log4j logging to async log4j logging mechanism.

We have decided on the following points for log4j to analyze the behavior of the system along with performance:

1) System throughput performance
2) Configuration of Log4j
3) Support to add logs in different storage e.g Files, Database asynchronously.
4) Support to add for Custom level. Need to add a new log level Audit.

Log4j supports async mode by the in-built async plugin. To enable it we need to add either existing and new appenders/loggers inside that async tag in log4j.xml
We have created Async appenders and added in appenders list inside log4j2.xml

<Async name="asyncAppender" level="info"> <AppenderRef ref="fileAppender"/> </Async>

As an async log4j implementation using lock-free structures, we have seen system performance increased to 25% in async with file appender only.
Testing system running 20 threads simultaneously creating different log level log-events at speed of 400 logs per second. Each log message size is around 4KB.

We have created two async appenders — One for file appender and one for console appender.
As the console is very slow in response, a console logger or appender should be run in an independent async mode without another appender, so that its slowness doesn’t impact other appenders.

No much change needed in the existing configuration of log4j to enable async mode.

We have also added a Custom level “AUDIT” and for enabling custom level in log4j2.xml we have added the following lines in log4j2.

<CustomLevels> <CustomLevel name="AUDIT" intLevel="150" /> </CustomLevels>

Level of the custom level set’s between error and fatal level.
To add custom ( AUDIT) log in code, we have used the following method
LOGGER.log(Level.getLevel(“AUDIT”), “Audit Message {}”, log);

To add error level logs in the database, we have again added async appender along with JDBC appender with a connection pooling system.
It is also running in different async threads.

Log4j documentation provides a detailed explanation of configuration parameters.
As the Log4j community added many improvements in recent years to increase the handling of logs, we have used log4j library version 2.13.1,

Originally published at https://www.neovasolutions.com on June 5, 2020.