Senior Software Developer, Big Data
AMER - Canada - Ontario - Toronto - University Ave
Autodesk
Job Requisition ID #
24WD77870Position Overview
Autodesk is looking for a Senior Software Developer, Big Data with experience building data pipelines to join our Enterprise Data Engineering & Visualisation (EDEV) team. The EDEV Team is chartered with building data pipelines and solutions that allow our users to obtain important business insights. These solutions serve as the building blocks for all data engineers, data scientists and data analysts throughout the organisation. Also, this team establishes and promotes data engineering best practices across Autodesk, leading efforts to make data an integral part of decision-making at Autodesk. You will build data solutions for complex data transformation pipelines, and lending technical expertise within the group.
You will report to Senior Software Development Manager based in the US. This role is based out of Toronto, in a hybrid mode. Autodesk supports FLEX working model.
Responsibilities
Automate data transformations against data sourced from a variety of systems including desktop, web, and mobile product usage logs and business systems data
Develop solutions applying CI/CD processes to orchestrate automated batch and NRT (Near Real Time) pipelines running PySpark and dbt data transformations using Apache Airflow
Use programming languages used for data manipulation, including SQL, Python, Spark, PySpark, Spark SQL, Java, Jinja
Design, develop, execute, and document software solutions to address complex data collection, processing, transformation, and testing
Collaborate with peer organisations, DevOps, support organisations on technical issues and help troubleshoot code level problems and performance issues
Recommend architectural standards, best practices and quality assurance processes for data related systems and applications
Build data quality and durability tracking mechanisms to provide visibility into and address changes in data ingestion, processing, and storage
Lead and/or help design data schemas to ensure common protocols and storage mechanisms are used across all data services
Partner with data and software architects to design data models, APIs or other architectural elements
Minimum Qualifications
5 years of experience in cloud ETL, ELT and near-real time data collection, transport, and processing technologies
3+ years of experience with data modeling, including the design of schemas optimized for data retrieval
5+ year of experience in programming with SQL, Python, Spark, PySpark, Spark SQL, Java, Jinja and related languages
3+ years of experience working with a big data environment including Hadoop, Hive, Spark and Presto
Experience creating detailed technical documentation which addresses business requirements
Experience architecting and implementing data testing solutions
Experience working with a big data environment including Hadoop, Hive, Spark and Presto
Proficiency in building data transformation and analysis using PySpark and dbt
Experience working with workflow management tools, like Airflow and Azkaban
Experience working with GitHub, Jira, and Jenkins
Familiarity with data governance frameworks, SDLC (Software Development Life Cycle), and Agile methodology
Strong technical and problem-solving skills and continuous interest in learning best of class technologies around data warehousing, data wrangling, data quality, data governance and data ethics
Learn More
About Autodesk
Welcome to Autodesk! Amazing things are created every day with our software – from the greenest buildings and cleanest cars to the smartest factories and biggest hit movies. We help innovators turn their ideas into reality, transforming not only how things are made, but what can be made.
We take great pride in our culture here at Autodesk – our Culture Code is at the core of everything we do. Our values and ways of working help our people thrive and realize their potential, which leads to even better outcomes for our customers.
When you’re an Autodesker, you can be your whole, authentic self and do meaningful work that helps build a better future for all. Ready to shape the world and your future? Join us!
Salary transparency
Salary is one part of Autodesk’s competitive compensation package. Offers are based on the candidate’s experience and geographic location. In addition to base salaries, we also have a significant emphasis on discretionary annual cash bonuses, commissions for sales roles, stock or long-term incentive cash grants, and a comprehensive benefits package.Diversity & Belonging
We take pride in cultivating a culture of belonging and an equitable workplace where everyone can thrive. Learn more here: https://www.autodesk.com/company/diversity-and-belonging
Are you an existing contractor or consultant with Autodesk?
Please search for open jobs and apply internally (not on this external site).
* Salary range is an estimate based on our AI, ML, Data Science Salary Index 💰
Tags: Agile Airflow APIs Azkaban Big Data CI/CD Data governance Data pipelines Data quality Data Warehousing dbt DevOps ELT Engineering ETL GitHub Hadoop Java Jenkins Jira Pipelines PySpark Python SDLC Spark SQL Testing
Perks/benefits: Career development Competitive pay
More jobs like this
Explore more AI, ML, Data Science career opportunities
Find even more open roles in Artificial Intelligence (AI), Machine Learning (ML), Natural Language Processing (NLP), Computer Vision (CV), Data Engineering, Data Analytics, Big Data, and Data Science in general - ordered by popularity of job title or skills, toolset and products used - below.
- Open Business Intelligence Engineer jobs
- Open Data Engineer II jobs
- Open Senior Business Intelligence Analyst jobs
- Open Data Science Intern jobs
- Open Junior Data Scientist jobs
- Open Lead Data Analyst jobs
- Open Business Intelligence Developer jobs
- Open Data Scientist II jobs
- Open Data Science Manager jobs
- Open Business Data Analyst jobs
- Open Sr. Data Scientist jobs
- Open Marketing Data Analyst jobs
- Open Principal Data Scientist jobs
- Open Research Scientist jobs
- Open Data Analytics Engineer jobs
- Open Sr Data Engineer jobs
- Open MLOps Engineer jobs
- Open Data Analyst Intern jobs
- Open Azure Data Engineer jobs
- Open Manager, Data Engineering jobs
- Open Product Data Analyst jobs
- Open Big Data Engineer jobs
- Open Data Engineering Manager jobs
- Open Junior Data Engineer jobs
- Open ETL Developer jobs
- Open GCP-related jobs
- Open Data quality-related jobs
- Open Java-related jobs
- Open ML models-related jobs
- Open Business Intelligence-related jobs
- Open Data management-related jobs
- Open Privacy-related jobs
- Open PhD-related jobs
- Open Data visualization-related jobs
- Open Deep Learning-related jobs
- Open NLP-related jobs
- Open PyTorch-related jobs
- Open Finance-related jobs
- Open TensorFlow-related jobs
- Open APIs-related jobs
- Open LLMs-related jobs
- Open Generative AI-related jobs
- Open Consulting-related jobs
- Open Snowflake-related jobs
- Open Hadoop-related jobs
- Open CI/CD-related jobs
- Open Data governance-related jobs
- Open Kubernetes-related jobs
- Open Airflow-related jobs
- Open Data warehouse-related jobs