Data Engineer

Company: Software Resources
Location: San Francisco
Posted on: April 1, 2025

Job Description:

Software Resources has an immediate job opportunity for a Data Engineer with a major corporation in San Francisco, CA (hybrid).Duration: 9 month(s)Pay Rate: $75 - $83/hr DOEFocus: SQL Knowledge, workflow management (Airflow tool), and Machine Learning Knowledge specifically feature. Understanding data inputs into the model and feature analysis. Feature Engineering. Training data set composition.Department Overview:Applied Machine Learning Engineers, Data Engineers, and Data Scientists on the Streaming Machine Learning and Innovation team develop and maintain recommendation and personalization algorithms for Streaming's suite of streaming video apps. As a member of this team, you will collaborate across Engineering, Product, and Data teams to apply machine learning methods to meet strategic product personalization goals, explore innovative, cutting-edge techniques that can be applied to recommendations, and constantly seek ways to optimize operational processes.Our team is---

• A group of engineers and data scientists with diverse expertise delivering solutions together.
• Collaborative and dynamic.
• Embracing agile practices.
• Using continuous integration/automated testing.
• Led by startup veterans.Required Qualifications:
• 2+ years of data engineering experience
• Deep knowledge of the Python data ecosystem
• Great coding and problem-solving skills
• Experience in building large datasets and scalable services
• Experience deploying and running services in AWS and in engineering big-data solutions using technologies like Databricks, EMR, S3, Spark, and Docker
• Excellent communication and people engagement skillsPreferred Qualifications:
• Experience with Pyspark to perform data transformations, feature engineering, data analysis, and ML training data generation
• Experience building streaming pipelines using Kafka, Spark, or Flink
• ML algorithmic and systems knowledge/experienceResponsibilities:
• Partner with technical and non-technical colleagues to understand ML algorithm feature, data, and workflow requirements.
• Implement necessary feature analysis, new features, training datasets and related workflows and monitoring metrics and services
• Work with Engineering teams to collect required data from internal and external systems
• Develop and maintain ETL routines using orchestration tools such as Airflow
• Collaborate with machine learning practitioners to design and build model and data forward solutions
• Deploy scalable streaming and batch data pipelines to support petabyte scale datasets
• Enforce common data design patterns to increase code maintainability
• Create ETL architecture designs and conduct reviews
• Perform ad hoc data analysis and maintenance as necessary
• Partner with team leads to identify, design, and implement internal process improvements
• Drive and maintain a culture of quality, innovation, and experimentation
• Work in an Agile environment that focuses on collaboration and teamworkRequired Education:Bachelor's degree or relevant years of work experienceAbout Software ResourcesSoftware Resources, founded more than 3 decades ago, is a trusted staffing partner specializing in Technology (IT, Creative, & Marketing), Finance, & Accounting placements. We work with companies nationwide that need skilled professionals and connect them with candidates like you looking for the right opportunity.We are always looking for professionals at all experience levels and in various specialties to meet our clients' hiring needs. As a member of our team, you will have access to benefits including medical, dental, and vision coverage, a 401(k) with company match, short-term disability, life insurance with AD&D.If this position interests you, we encourage you to apply. You can view all our job openings at Software Resources Careers.
  #J-18808-Ljbffr

Keywords: Software Resources, San Francisco , Data Engineer, Engineering , San Francisco, California