Big Data Engineer

Location: Hyderabad

What You’ll Do:

  • Understand business requirements of different data sources
  • Work on the migration of data across Hadoop clusters
  • Retrieve data of different formats (structured/semi-structured /unstructured), find data anomalies and correct if required
  • Work with theData Science team to prepare data for analytics
  • Prepare Hive automation script to perform ETL processing
  • Handle XML data and prepare denormalized table on this data
  • Develop a Hive script to maintain the audit details
  • Write complex Hive queries for data analysis as per the business requirement
  • Coordinate with the onsite team to define work scope, identify impact(s) from changes
  • Actively participate in the regular sprint ceremonies
  • Work with CI/CD team to push code changes - execute dry runs, ensure code changes are reflected
  • Coordinate with the onsite team for code review and implement any code review feedback
  • Attach test cases with test results
  • Maintain code repository with appropriate comments
  • Create change request for code push to production, support deployment and issues that arise
  • Work on POC tasks of different backend technology stacks as needed
  • Monitor the production job execution and initiate appropriate action if required

What You Know:

  • 4-8 years of hands-on experience working with Hadoop distribution platforms such as  HortonWorks, Cloudera, and MapR
  • Experience with big data management, design, implementation, and deployment in big data platforms
  • Take end-to-end responsibility of Hadoop life cycle in the organization
  • How to code in multiple programming languages; Java, Python, Perl, Ruby, etc.
  • Hands-on experience with big data technologies specifically Hadoop MapReduce, HBase, Pig, Hive, Impala, Spark Scala/Python, Sqoop, Flume and Kafka
  • Strong experience with data warehousing, ETL implementation, and hands-on experience with any related tool
  • Experience with SQL and NoSQL databases
  • Must have cloud experience: Azure / AWS / Google
  • Effective communication of complex technical concepts to broad audiences

Education:

  • Bachelor’s degree in a specific technical field such as Computer Science, Mathematics, or Statistics or equivalent work experience; Master’s degree a plus

 

Ready to apply? Email your resume to recruitment_hyderabad@nisum.com.