Big Data Developer

Big Data requirement 4-9yrsProficient in Core JAVA (Multithreading and Collections framework)Proficient in cleaning, transforming, and analyzing vast amounts of raw data from various systems using Spark to provide ready-to-use data to our feature developers and business analysts.Proficient in designing efficient data processing pipelines by writing suitable Spark batch and streaming Jobs involving various transformations and aggregations.Proficient in Spark data structures and APIs based on RDDs, Dataframe and Datasets.Proficient in Spark query tuning and performance optimizationProficient in usage of Hadoop Distributed File System (HDFS)Proficient in data formats such as JSON, ORC, Parquet, Avro, etc.Desirable: Understanding of distributed systems (e.g. CAP theorem, partitioning, replication, consistency, and consensus)Desirable: Working with unstructured database (Mongo DB, HBase, Cassandra)Desirable: Understanding and experience of working on SQL databases, such as Oracle, MySQL or PostgresDesirable: Experience with Zookeeper, Hive, Sqoop and other utilities of Hadoop Ecosystem.Must have strong troubleshooting skill in debugging the failure of Spark Jobs on production cluster. Contact - 8861570531


Key Skills
C++, Linux, Multithreading., Socket Programming

Job Summary

  • Published on: NA
  • Salary: NA
  • Location: Mumbai