API for Reactive Log Processing with focus on AWS!ΒΆ

Lumber-Mill is designed for programmers/devops/SRE etc with a professional programming background but with an emergent interest for devops, monitoring and log processing and who want total control of the event pipeline.

Benefits

  • Deployed as AWS Lambdas
  • API instead of runtime - complete java8 api
  • No need for plugins, just depend on any third-party library you might need
Observable call(Observable eventStream) {

    // Parse and de-normalize events
    eventStream.compose ( new CloudWatchLogsEventPreProcessor())

    .flatMap (
        grok.parse (
           field:        'message',
           pattern:      '%{AWS_LAMBDA_REQUEST_REPORT}'))

    .flatMap ( addField('type','cloudwatchlogs'))

    .flatMap ( fingerprint.md5())

    .buffer (100)

    .flatMap (
        AWS.elasticsearch.client (
            url:          '{es_url}',
            index_prefix: 'lumbermill-',
            type:         '{type}',
            region:       'eu-west-1',
            document_id:  '{fingerprint}'
        )
    )
}

Contents: