Please answer the following questions in order to process your application.
Select your working status in the UK *
Describe your level of Security clearance in the UK *
File Attachments:
(2MB file maximum. doc, docx, pdf, rtf or txt files only)
* denotes required field
Additional Information:
Availability/Notice
Hourly Rate GBP
Approximately how far are you willing to travel to work (in miles) ?
Key Privacy Information
When you apply for a job, TopTechJobs will collect the information you provide in the application and disclose it to the advertiser of the job.
If the advertiser wishes to contact you they have agreed to use your information following data protection law.
TopTechJobs will keep a copy of the application for 90 days.
More information about our Privacy Policy.
Job Details
(SC cleared) data engineer - AWS/Spark (Contract)
Location: Bristol Country: UK Rate: Up to £450 per day
Daily rate: £450 outside IR35
Hybrid: 2days/w in Bristol
The job:
The role will require support the lead data engineer in the analysis and process of the raw platform data and additionally working with the lead data architect to design an ETL pipeline to a cloud data platform. We are looking for a candidate who understands and knows how to code using Spark technology using pySpark as a minimum. Experience of a cloud infrastructure will be a bonus.
Additionally the role will require the support of stakeholder engagement of the source systems in various workshops to understand extant data quality issues and to play back solutions that we are developing for stakeholder endorsement. You will be supported with a Business Analysts to bring all of this together.
Responsibilities
- Analyze and organize raw platform and data warehouse data
- Build data systems and pipelines in AWS, Delta
- Evaluate business needs and objectives with Data Architect
- Interpret trends and patterns
- Conduct complex data analysis and report on results
- Assist in the data lineage tasks
- Assist in the empirical data analysis of source and data warehouse data sets
- Build prototypes
- Combine raw information from different sources
- Explore ways to enhance data quality and reliability using AWS tooling
- Identify opportunities for data acquisition
- Develop analytical tools and programs
- Collaborate with stakeholders and architects
- Requirements analysis with BA.
Requirements and skills
- Previous experience as a data engineer or in a similar role
- Technical expertise with data models, data mining, and segmentation techniques
- Knowledge of programming languages (eg Java and Python, Scala, R, SQL)
- Hands-on experience with SQL database design
- Great numerical and analytical skills
- Degree in Computer Science, IT, or similar numeric field
- Data engineering certification (eg IBM Certified Data Engineer) is a plus
Posted Date: 03 Apr 2024
Reference: JSSCDESHAO
Employment Business: Methods Business and Digital Technology Limited
Contact: Shao Fan