May 26, 2023
Apply Now
Phoenix, AZ, United States
May 26, 2023
Apply NowJob description
We are seeking a talented and experienced Data Engineer to join our dynamic Infrastructure Team. As a vital member of our team, you will play a strategic role in expanding, refining, and advancing our infrastructure utilizing cutting-edge technologies like Amazon Web Services (AWS). Join us in shaping the future of our data analytics and engineering tools, and help us scale our work statewide to empower K-12th grade students, parents, and educators through our innovative higher education planning platform.
Qualifications
Proficiency in SQL (Required: 2 years)
Strong knowledge of cloud infrastructure, particularly AWS (Required: 2 years)
Must have US work authorization
2 years of experience as a Data Engineer
Remote/Virtual - Candidates must reside in California
The ideal candidate for this role is a seasoned professional with a proven track record in designing and building complex data architectures. You possess the expertise to test, maintain, and refine data pipelines while effectively scaling data intake to handle terabytes and petabytes of information. Your skills in optimizing data delivery and automating manual processes are second to none. Comfortable with both structured and unstructured data, you excel in troubleshooting data loading and processing tools using SQL, Python, shell scripting, and AWS. Furthermore, you're ready to take on leadership responsibilities for special projects when needed.
We place a strong emphasis on robust data documentation and data governance protocols, so extensive experience in these areas is essential. Although this position does not involve direct supervisory responsibilities, you will actively collaborate and partner with other subject matter experts on a project management basis. You thrive in leading and managing work with many unknowns, embracing ambiguity and finding innovative solutions. Your deep understanding and curiosity about the needs and behaviors of students, educators, and parents, combined with your passion for educational equity, make you an invaluable asset to our team.
What Will You Be Doing
Manage, refine, and enhance our AWS cloud services infrastructure, including EC2, VPC, RDS, ECS, CloudWatch, CloudFormation, CloudTrail, transfer for sFTP, S3, Lambda, Secrets Manager, and Route 53.
Lead ETL/ELT processes by developing, refining, and implementing data loading and processing tools, such as Snowflake, Airflow, Python, SQL, dbt, and shell scripts.
Review, redesign, and expand our existing analytics and data processing architecture to create optimal data pipelines.
Identify, design, and implement internal process improvements, including automating manual processes, optimizing data delivery, and designing scalable and automated infrastructure.
Maintain and update documentation of data architecture at both macro and micro levels, ranging from architectural diagrams to script-level tasks in Airflow.
Lead meetings, conduct research, collect data, and analyze information.
Collaborate with key stakeholders on your projects.
Develop and maintain expert knowledge of our platform and organization.
Continuously improve data pipelines and architecture by staying updated on industry trends and best practices.
Necessary Technical Skills
Proficiency in building processes supporting data transformation, data structures, metadata, dependency, and workload management.
Advanced proficiency with cloud analytical tools such as Snowflake, Redshift, Hadoop, Spark, and Kafka.
Advanced proficiency with ETL tools like Matillion, dbt, Talend, and more.
Advanced proficiency with data pipeline and workflow management tools such as Airflow, Azkaban, Luigi, and others.
Strong experience building and optimizing data pipelines, cloud architectures, and data sets.
Advanced scripting language skills, including Python, R, Scala, etc.
Proficiency in developing cloud infrastructure in AWS or currently pursuing AWS Certification as a Solutions Architect - Associate.
Expertise in SQL (SQL Server, PostgreSQL, MySQL, etc.) and understanding of relational databases, query authoring and optimization, as well as working familiarity with various databases.
Proven success in manipulating, processing, and extracting value from large structured and unstructured datasets.
Your Strengths
Strong decision-making skills and collaborative spirit, with the ability to transform abstract brainstorming into actionable proposals.
Self-driven and capable of advancing projects with minimal supervision, while providing colleagues with actionable proposals to advance collective efforts.
Thrive in a fast-paced environment with changing priorities and deadlines.
Exceptional multitasking abilities, effortlessly managing multiple projects of varying scopes.
Meticulous attention to detail.
Excellent verbal communication skills, enabling effective communication with professionals at all levels.
Strong organizational, project management, and time management skills.
We are committed to fostering an environment of mutual respect, where equal employment opportunities (EEO) are available to all employees and applicants. We do not discriminate on the basis of race, color, ancestry, national origin, genetic characteristics, sex, gender identity, gender expression, sexual orientation, marital/parental status, political affiliation, religion, age, disability, pregnancy, childbirth, breastfeeding, or veteran status. In addition to federal law requirements, we comply with applicable state and local laws governing nondiscrimination in employment. Our workplace policies and hiring practices align with federal, state, and local law. We are interested in hiring qualified candidates who are eligible and authorized to work in the United States. Please note that we are unable to sponsor visas at this time, and therefore, we cannot consider applicants who currently or in the future require immigration sponsorship for work authorization (e.g., H1B or F1 Student Visa).
Employment Type: Full-Time
Salary: $ 110,000.00 160,000.00 Per Year
Qualifications
Proficiency in SQL (Required: 2 years)
Strong knowledge of cloud infrastructure, particularly AWS (Required: 2 years)
Must have US work authorization
2 years of experience as a Data Engineer
Remote/Virtual - Candidates must reside in California
The ideal candidate for this role is a seasoned professional with a proven track record in designing and building complex data architectures. You possess the expertise to test, maintain, and refine data pipelines while effectively scaling data intake to handle terabytes and petabytes of information. Your skills in optimizing data delivery and automating manual processes are second to none. Comfortable with both structured and unstructured data, you excel in troubleshooting data loading and processing tools using SQL, Python, shell scripting, and AWS. Furthermore, you're ready to take on leadership responsibilities for special projects when needed.
We place a strong emphasis on robust data documentation and data governance protocols, so extensive experience in these areas is essential. Although this position does not involve direct supervisory responsibilities, you will actively collaborate and partner with other subject matter experts on a project management basis. You thrive in leading and managing work with many unknowns, embracing ambiguity and finding innovative solutions. Your deep understanding and curiosity about the needs and behaviors of students, educators, and parents, combined with your passion for educational equity, make you an invaluable asset to our team.
What Will You Be Doing
Manage, refine, and enhance our AWS cloud services infrastructure, including EC2, VPC, RDS, ECS, CloudWatch, CloudFormation, CloudTrail, transfer for sFTP, S3, Lambda, Secrets Manager, and Route 53.
Lead ETL/ELT processes by developing, refining, and implementing data loading and processing tools, such as Snowflake, Airflow, Python, SQL, dbt, and shell scripts.
Review, redesign, and expand our existing analytics and data processing architecture to create optimal data pipelines.
Identify, design, and implement internal process improvements, including automating manual processes, optimizing data delivery, and designing scalable and automated infrastructure.
Maintain and update documentation of data architecture at both macro and micro levels, ranging from architectural diagrams to script-level tasks in Airflow.
Lead meetings, conduct research, collect data, and analyze information.
Collaborate with key stakeholders on your projects.
Develop and maintain expert knowledge of our platform and organization.
Continuously improve data pipelines and architecture by staying updated on industry trends and best practices.
Necessary Technical Skills
Proficiency in building processes supporting data transformation, data structures, metadata, dependency, and workload management.
Advanced proficiency with cloud analytical tools such as Snowflake, Redshift, Hadoop, Spark, and Kafka.
Advanced proficiency with ETL tools like Matillion, dbt, Talend, and more.
Advanced proficiency with data pipeline and workflow management tools such as Airflow, Azkaban, Luigi, and others.
Strong experience building and optimizing data pipelines, cloud architectures, and data sets.
Advanced scripting language skills, including Python, R, Scala, etc.
Proficiency in developing cloud infrastructure in AWS or currently pursuing AWS Certification as a Solutions Architect - Associate.
Expertise in SQL (SQL Server, PostgreSQL, MySQL, etc.) and understanding of relational databases, query authoring and optimization, as well as working familiarity with various databases.
Proven success in manipulating, processing, and extracting value from large structured and unstructured datasets.
Your Strengths
Strong decision-making skills and collaborative spirit, with the ability to transform abstract brainstorming into actionable proposals.
Self-driven and capable of advancing projects with minimal supervision, while providing colleagues with actionable proposals to advance collective efforts.
Thrive in a fast-paced environment with changing priorities and deadlines.
Exceptional multitasking abilities, effortlessly managing multiple projects of varying scopes.
Meticulous attention to detail.
Excellent verbal communication skills, enabling effective communication with professionals at all levels.
Strong organizational, project management, and time management skills.
We are committed to fostering an environment of mutual respect, where equal employment opportunities (EEO) are available to all employees and applicants. We do not discriminate on the basis of race, color, ancestry, national origin, genetic characteristics, sex, gender identity, gender expression, sexual orientation, marital/parental status, political affiliation, religion, age, disability, pregnancy, childbirth, breastfeeding, or veteran status. In addition to federal law requirements, we comply with applicable state and local laws governing nondiscrimination in employment. Our workplace policies and hiring practices align with federal, state, and local law. We are interested in hiring qualified candidates who are eligible and authorized to work in the United States. Please note that we are unable to sponsor visas at this time, and therefore, we cannot consider applicants who currently or in the future require immigration sponsorship for work authorization (e.g., H1B or F1 Student Visa).
Employment Type: Full-Time
Salary: $ 110,000.00 160,000.00 Per Year
