Pipeline reliability review
Retries, duplicates or recovery are difficult to trust.
How I can help with this →
Hi, I’m
Alex Tarverdy
Senior Data Engineer Scala Developer
I’m the engineer you can call when a pipeline is unreliable, a Spark job is slow, or a Scala project needs another pair of hands. I help you understand the problem and build a next step you can trust.
Reliable data.
Considered engineering.
A pipeline that needs a second look?
A data project that needs another pair of hands?
I work with Scala, Spark, Kafka and Databricks, with experience across banking ingestion and financial reporting. I can help you investigate a problem, agree a practical scope and build the next step.
How I can help
Start with the problem
you need to solve.
Spark performance investigation
Spark jobs are slow, resource hungry or unstable.
How I can help with this →Data correctness review
The pipeline finishes, but the numbers do not add up.
How I can help with this →Scala implementation support
Your team needs help turning a data problem into working software.
How I can help with this →Selected projects
Look inside the work.
These are things I’ve built and explored. Open a project to see the code, the decisions and what I’ve checked.
SQL Lineage Explorer
Trace output columns through chained SQL queries, inspect intermediate expressions and explore lineage in a web interface.
Python / Spark SQL parsing / Flask
Financial transaction reporting with Spark
A financial reporting code sample with DataFrame and SQL implementations, receiver checks, currency conversion and guarantor aggregation.
Scala 2.13 / Spark 3.5.1 / PostgreSQL
Versioned CSV ingestion and a MongoDB query API
A Scala prototype connecting versioned CSV feed files to MongoDB and exposing ID/date-range queries through http4s.
Scala 2.12 / Cats Effect 3 / http4s
Relevant experience
Banking data.
Practical engineering.
My work at blu Bank and on a Santander engagement spans ingestion, reporting transformations, validation and developer tooling.
When successful pipelines produce incorrect data.
Engineering for correctness in financial regulatory reporting: transformations, validations and the business meaning behind the output.
Spark / SQL / Databricks
Keeping banking data moving.
Real-time consumers, incremental ingestion and orchestration across relational and document databases at blu Bank.
Scala / Python / Kafka
Making data engineering easier to deliver.
CI improvements, extraction configurations, internal tools and knowledge sharing that support day-to-day engineering.
CI / Configuration / Documentation
Latest writing
What I’m thinking through.
A pipeline can be green and still be wrong.
Why successful execution is only one part of trusting a data pipeline.
Retries need a destination that can handle them.
What retries and primary keys can guarantee—and where the boundaries matter.
Let’s untangle your data problem.
Tell me what you’re building, what’s getting in the way and where you need another pair of hands.