Skip to main content

Modern Data Stack Engineering

Comprehensive coverage of building, optimizing, and scaling modern data architectures using cloud-native tools like Databricks, Snowflake, and dbt.

The articles and projects here focus on practical implementation—medallion architecture patterns, cost optimization, pipeline design, and choosing between platforms like Databricks and Snowflake. If you're building or evolving a data platform, this is where to start.

Articles

Beyond the Spin: Building a Creative Loading Bar in Streamlit for Better UX

When Databricks queries take 5–20 seconds, a static orange banner feels like an error. Swap in a branded Automower animation with the same Streamlit context-manager API.

LINHAC 2026 — Conference Report

LINHAC 2026 confirmed a clear shift: hockey analytics is becoming core infrastructure, not a side project. This report summarizes the strongest research signals, practical club examples, and what organizations should prioritize next.

When to Use a Consultant Instead of Hiring: A Data Engineering Perspective

· Updated
Hiring a senior data engineer costs €120–200k the first year. But for bounded projects — Databricks buildouts, dbt migrations, lakehouse design — consulting delivers faster and cheaper. Here is when each makes sense.

22 Articles in 26 Days: Session Data, Keywords, and What the Automation Actually Produces

· Updated
Four weeks of build in public: 22 n8n tutorials published, 951 sessions, 38 ranked keywords, and 2 top-10 rankings — without a single external backlink. Here are the numbers, without spin.

n8n Split in Batches Node: Processing Large Datasets Without Hitting Rate Limits

· Updated
When you need to process thousands of records in n8n, feeding them all at once into an API or database crashes the workflow or triggers rate limits. The Split in Batches node is how you process large datasets reliably — here is how it actually works.

Agentic Data Organisation: How I Set Up an AI-Driven Data Organisation with Claude

· Updated
A practical walkthrough of my agentic data organisation: roles, ownership, agent coordination, and how Claude drives decision flow across analytics, platform, and governance.

n8n Merge Node Modes Explained: Combine, Multiplex, and Pass-Through Under Partial Input

· Updated
The n8n Merge node has four modes with subtle behavior differences that only surface with partial input, mismatched item counts, or timing gaps between branches. Here is what each mode actually does.

n8n Error Trigger: Workflow-Level Error Catching and When It Actually Fires

· Updated
The n8n Error Trigger node is a workflow that runs when another workflow fails — but the when and why are full of edge cases. Here is how it actually works, what it misses, and how to build reliable error handling in production n8n.

From Raw Data to Revenue Decisions: Building an End-to-End Analytics Platform

How I designed and shipped a full-stack data intelligence system for a global manufacturing company — from Databricks pipelines to AI-powered dashboards.

Designing AI Agent Architectures for Modern Data Engineering Workflows

Data Engineering Consulting: Building Scalable, Future-Proof Data Solutions

Master n8n: Building Robust Data Pipelines with Workflow Automation – A Step-by-Step Guide

From BI Reporting to Data-Driven Automation: A 10-Year Evolution of Data Strategy

How data strategy evolved from static BI reports to AI-powered automation. Learn from 10 years of data engineering experience: from centralized reporting to data mesh, and why making AI investments pay for themselves requires collapsing legacy and modern stacks.

Mastering Low-Code Automation: A Developer's Guide

## Introduction to Efficient Workflow Management

n8n vs Zapier vs Make.com vs sim.ai: Which Workflow Automation Tool is Best in 2025?

Comprehensive comparison of the top workflow automation platforms: n8n, Zapier, Make.com, and sim.ai. Find the perfect tool for your automation needs with real-world examples, pricing analysis, and expert recommendations from a data engineer.

Automating Your Freelance Workflow: Practical Examples and Tools

As a freelancer, managing your time and workload efficiently is crucial for delivering high-quality work and growing your business. These examples show practical automations that save time.

Automating Workflows with Modern Tools: A Data Engineer's Perspective

As an agentic engineer, I've witnessed the significant impact of workflow automation on a business's bottom line. By streamlin...

Unlocking Efficiency: A Beginner's Guide to Developer Automation Toolkit

## Introduction to Automation Toolkits As an agentic engineer with hands-on experience in streamlining workflows, I've seen th...

API Automation for Startups: Unlocking Efficiency with Modern Tools and Expertise

As an agentic engineer, I've had the privilege of working with numerous startups in Stockholm, helping them streamline their o...

Automating Personal Data Workflows: A Practical Guide

As an agentic engineer, I've worked with numerous organizations in Stockholm, helping them streamline their data processes and...

Beyond Dashboards: How to Build Living Data Experiences with Conversational Analytics

Dashboards fail because they require users to go somewhere else. Data prototyping expertise: learn how proactive insights and conversational analytics replace traditional dashboards with living data experiences.

Integrating APIs with n8n: A Simplified Approach to Workflow Automation

As an agentic engineer, I've worked on numerous projects that involve integrating multiple APIs to create seamless workflows. ...

Boosting GitHub Workflow: Automation Best Practices for Efficient Development

As an agentic engineer, I've seen firsthand how automating repetitive tasks can significantly improve development efficiency a...

Unlocking Efficiency: The Role of a Digital Productivity Consultant

As an agentic engineer, I've seen firsthand how inefficient workflows can hinder even the most talented teams. In today's fast...

Mastering Custom Automation Scripting for Enhanced Efficiency

As an agentic engineer, I've seen firsthand the impact that custom automation scripting can have on streamlining workflows, re...

Unlocking Efficiency: Automation Tooling Strategist Best Practices

As an agentic engineer, I've seen firsthand the impact that efficient automation tooling can have on a project's success. In t...

Boosting Business Efficiency with Automation Freelancer Expertise

As an agentic engineer based in Stockholm, Sweden, I've seen firsthand how automation can transform businesses. In today's fas...

Accelerating Outside-In Development: AI-Powered Multi-Market Feedback Collection with OutSystems

How I built a global feedback portal using OutSystems low-code platform, ChatGPT API for translations, and modern data stack to enable rapid customer-centric development across multiple markets.

Modern Data Stack 2025: Complete Guide for $10/Month | Data Engineering Solutions

Learn to build a complete modern data stack for only $10/month. Comprehensive guide with practical examples from professional data engineering services. Including Snowflake, dbt, and cloud-native tools.

Why Most Business Dashboards Fail (Enterprise, Small Business & Sports Teams) - Data Engineer Insights

Data engineer Emil Ingemar Karlsson explains why 60% of dashboards fail and shares proven strategies for creating effective data visualizations that drive real business results from his experience in Nacka, Sweden.

Projects

Enterprise ML & Data Platform — 100 Jobs, 631 Tables, NLP at Scale

End-to-end data and ML platform for a global manufacturing company. 100 Databricks jobs, 631 Unity Catalog tables, support intelligence across 1.46M cases, production churn models, demand forecasting, and a 17-page Streamlit dashboard — built and operated solo over three years.

End-to-End Analytics Platform — Databricks, Streamlit & GPT-4o

Full-stack data intelligence system for a global manufacturer. Medallion architecture on Databricks, AI-powered natural language querying, z-score anomaly detection, and 10 production dashboards covering 14 markets — shipped in 12 weeks.

Conversational Analytics Prototype - Beyond Dashboards

Interactive prototype demonstrating the future of data analytics: natural language queries, proactive insights, and zero-training data exploration. Built to show what comes after traditional dashboards.

AI-Powered Development Workflow - Claude MCP & React Prototype Platform

Comprehensive AI development workflow using Claude Sonnet 4 through MCP server, connecting multiple data sources to generate functional React prototypes from strategy to deployment in days.

Automated Hockey Analytics Pipeline - AI & LLM Integration with Mage AI Data Processing

Fully automated hockey analytics platform combining AI, LLMs, and advanced data engineering. Complete pipeline automation using Mage AI, DuckDB, and interactive dashboards for comprehensive sports analysis.

Multilingual Survey Analytics - AI-Powered NLP for Customer Insight Intelligence

Advanced multilingual survey analysis platform using AI translation and NLP categorization. Comprehensive customer insights solution leveraging Databricks, Power BI, and machine learning for survey intelligence.

AI-Powered Customer Support Analytics - Large Language Models for ServiceNow Intelligence

Advanced AI and LLM integration for customer support optimization using ServiceNow data transformation. Intelligent ticket categorization and customer insights platform specializing in AI analytics.

Customer-Centric B2B Platform Strategy - Outside-In Data Analytics for Investment Decisions

Strategic B2B platform transformation using customer-centric data analytics and user insights mapping. Advanced decision-making framework using Databricks, Power BI, and Python for business intelligence solutions.

Unified Data Ecosystem Architecture - Enterprise Data Lake & Business Intelligence Platform

Comprehensive enterprise data ecosystem with scalable data lake architecture using Databricks and Power BI. Advanced data engineering and business intelligence solution delivering actionable insights and data governance.

Modern Data Stack Architecture - Cost-Effective Cloud Infrastructure for $10/Month

Complete guide to building scalable modern data stack using cloud-native tools and serverless architecture. Affordable data engineering solution demonstrating efficient data processing, automation, and visualization.

Customer-Focused KPI Analytics Platform - Real-Time Performance Intelligence & Data Integration

Advanced KPI tracking and management platform with real-time alerts and comprehensive data integration. Customer experience analytics solution using Snowflake, Databricks, and Power BI technologies.

Hockey Analytics Dashboard - Advanced Sports Data Visualization & Performance Intelligence

Comprehensive hockey performance analytics platform using Azure cloud infrastructure and Python data engineering. Advanced sports analytics dashboard specializing in hockey insights and performance metrics.

Customer Support Optimization - Advanced Analytics for Enhanced Service Experience

Comprehensive customer service operations transformation using business intelligence and data analytics. Support process optimization and performance enhancement, specializing in ServiceNow, Power BI, and customer experience analytics.

Data-Driven IT Operations Optimization - Advanced Analytics for Proactive Service Management

Enterprise IT operations transformation using advanced data analytics and business intelligence. Historical data pattern analysis for proactive optimization, specializing in ServiceNow, Power BI, and SQL analytics solutions.

Food Waste Reduction Platform - Django Web Application for Sustainable Commerce

Innovative Django-based platform connecting consumers with surplus food from restaurants and grocery stores. Full-stack development solution using Python, Django, and Bootstrap for sustainable commerce applications.

Related Topics

Workflow Automation & Orchestration

Master n8n, sim.ai, and workflow orchestration for business process automation.

Hockey Analytics & Sports Data Science

AI-powered hockey analytics and advanced sports data science.

AI Agents & LLM Integration

AI agents, LLM integrations, and prompt engineering for intelligent solutions.

All Topics

Modern Data Stack Engineering

Build and optimize modern data architectures with cloud-native tools.

Workflow Automation & Orchestration

Master n8n, sim.ai, and workflow orchestration for business process automation.

Hockey Analytics & Sports Data Science

AI-powered hockey analytics and advanced sports data science.

Full-Stack Development (Astro/TypeScript)

Modern web development with Astro, TypeScript, and performance optimization.

Developer Productivity & Tooling

Developer workflows, tooling, and productivity patterns for technical professionals.

Stockholm/Nordic Data Engineering Market

Stockholm tech scene, Nordic data engineering market, and local insights.

Conversational Analytics & Next-Gen BI

Conversational analytics, natural language interfaces, and next-gen BI.

AI Agents & LLM Integration

AI agents, LLM integrations, and prompt engineering for intelligent solutions.