Danilo's Deck ⚡️
Profile
I'm a Data+AI Leader that owns delivery and technical standards for a distributed data engineering squad while staying hands-on, reviewing architecture and code on active workstreams and unblocking engineers day to day. My core stack today is the Databricks ecosystem (AI Agents, PySpark, Delta Lake, Unity Catalog), SQL, and the Azure cloud stack (Blob/ADLS Storage, Key Vault, Resource Groups, and other core services), provisioned and managed with Terraform. Python is my main programming language. Earlier in my career (5+ years ago) I also worked with Hadoop, Hive, HDFS, MongoDB, Tableau, Shell Script, JavaScript and Scala. My expertise spans across diverse industries such as customer behavior analysis, finance, energy, cosmetics, and insurance sector. I believe success in data engineering comes from empowerment, collaboration, and constant improvement.
Experience

Worldpanel

Full-time Remote · 5 yrs 9 mos (Current)

Data Engineering Tech Manager

Jul 2023 - Present · 3 yrs 3 mos | São Paulo e Região, Brazil

Working as technical manager who is capable to align expectations with the business stakeholders and coordinate tasks with the tech team. Currently leading a multidisciplinary and remote squad spread in different locations across the globe with data engineers, data testers, and frontend developers. Also working in pair with project manager, data architects, project owner and the head of data. Managing a global data-intensive platform for delivering aggregated data in different countries such as France, Great Britain, Spain, Portugal. Acting as the leader of the best practices in the data engineering domain to ensure robustness and scalability of the platform.

Senior Big Data Engineer

Jan 2022 - Jul 2023 · 1 yr 7 mos | Lisboa, Portugal

Working in a dedicated data processing platform handling almost 1PB data. Main focus in data processing of customer behaviour on supermarket shops. Handling scalability challenges either on stream as batch jobs. Supporting demands of different teams as business analysts, data science and reporting tool.

Senior Big Data Engineer - Consultant (via Integer Consulting)

Jan 2021 - Jan 2022 · 1 yr 1 mo | Lisboa, Portugal

The main project was focused on data processing of customer behaviour on supermarket shops. I have done Big Data solutions to support client demands. Worked on modern big data stack including Spark, Delta Lake and Azure.

Itaú Unibanco

3 yrs 2 mos

Senior Big Data Engineer (Tech Lead)

Jan 2020 - Dec 2020 · 1 yr | Brazil

Supervised a team of 6 technical analysts about solutions design. Worked closely with team members, stakeholders, and solution architect. Ensured architecture met business requirements. Worked on the evolution of the Itaú Unibanco data pipeline.

Senior Big Data Engineer

Oct 2019 - Dec 2019 · 3 mos | Brazil

Transitioned into a senior role, supporting the evolution of the Itaú Unibanco data pipeline ahead of the move into the Tech Lead position.

Big Data Engineer

Nov 2017 - Sep 2019 · 1 yr 11 mos | São Paulo e Região, Brazil

Built batch service for data delivery and reprocessing of Mainframe's data with linux component in shell script. Worked with data and analytics experts to strive for greater functionality in our Hadoop Data Lake. Support users in the consumption of data with appropriate tool like Hive, Impala, Alteryx and others. Ensured the data governance with physical models and metadata.

Accenture

2 yrs 1 mo

Digital Analyst

Aug 2015 - Aug 2017 · 2 yrs 1 mo | São Paulo e Região, Brazil

Worked in a big cosmetic industry in Brazil to build a data intensive application. Built Streaming and Batch services for processing data from sales advisor users using Kafka, Spark and Scala. Improvement performance of existing ETL processes and SQL queries for business area.

Certifications
  • Spark Award - Highly Commended — Insights (Issued Dec 2024 · Credential ID 125978635)
  • Growing Exceptional Managers Development Programme — Insights (Issued Oct 2024 · Credential ID 118612830)
  • Academy Accreditation - Databricks Lakehouse Fundamentals — Databricks (Issued May 2022 · Credential ID 51487740)
  • Academy Accreditation - Delta Lake Essentials — Databricks (Issued Jan 2021 · Credential ID 27832873)
  • Academy Accreditation - Unified Data Analytics Essentials — Databricks (Issued Jan 2021 · Credential ID 27657004)
Languages
  • Portuguese - Native
  • English - Full Professional
  • Spanish - Elementary
Education

Master's Degree in Computer Science (incomplete)

2013 to 2015 at Federal University of São Carlos, São Carlos, Brazil

Degree in System Analysis and Development

2010 to 2012 at Federal Institute of Education, Science and Technology of São Paulo, São Paulo, Brazil

Skills
  • Databricks
  • PySpark
  • Delta Lake
  • Unity Catalog
  • SQL
  • Python
  • Terraform
  • Azure Storage
  • Azure Key Vault
  • Azure Resource Groups
  • ETL/ELT
  • AWS
  • AWS Lambda
  • Scala
  • Shell Script
  • JavaScript
  • NoSQL Data Storages
  • Data Visualization
  • API Management