Our client, a top tier management firm is looking to engage with a Data Engineer.
Key Responsibilities -
1. Creating complex data processing pipelines, as part of diverse, high energy teams
2. Designing scalable implementations of the models developed by our Data Scientists
3. Hands-on programming based on TDD, usually in a pair programming environment
4. Deploying data pipelines in production based on Continuous Delivery practices
5. Advising clients on the usage of different distributed storage and computing technologies from the plethora of options available in the ecosystem.
Note - This is a 2 weeks project requiring full-time support based near Vijaywada, Andhra Pradesh wherein the flybacks and accommodation will be provided.
1. Python with experience in libraries like Numpy, NLTK, Pandas, etc.
2. Experience in data pipeline and workflow management tools (like Airflow) is a must
3. Experience with both RDBMS and NoSQL (MySQL, MongoDB hands on experience is a must, rest desirable)
4. Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases
5. Strong analytic skills related to working with unstructured datasets
6. Build processes supporting data transformation, data structures, metadata, dependency and workload management