Big Data/Java Developer

Job Overview

Big Data Java Developer, Manassas, VA Requirements Working with application development teams who write applications which generate logs, and with Operations teams who operate those applications, and through development of an understanding of the business of SWIFT customers, the successful candidate will drive and oversee the following Understanding of content and value of logs generated by messaging or service management tools Creation of syslog-ng or Filebeat configuration to transport logs from generating system into pipeline Development of parser(s) for the logs, using logstash GROK or custom parser developed in Java Development of configuration for a generic metrics engine defining fields to be used for calculating metrics of interest e.g. transit times Development of dashboards for useful visualization of insights found in those logs ndash Kibana mostly Working with development operations staff, conduct end-to-end tests of the entire pipeline from generating system to dashboard Use predictive analytics classification algorithm (like linear, logistic, random forest, SVM, TF-IDF etc) to predict Desire LanguagesTechnologies ndash Java, Python, Ansible, Logstash, Flume, Fluentd, Elastic Search, Kibana, Syslog-NG, Filebeat, Metricbeat, Data Science (Some knowledge) SkillsRequirements Python Java Logstash Elastic Search Apache Spark Data Analysis Software Engineering Big Data Infrastructure