Visual Flow

GET STARTED. Build ETL Pipelines Today

An ETL tool powered by Apache Spark on Kubernetes

We developed Visual Flow as a low-code solution for developers looking to leverage the ETL benefits of Spark without learning a programming language.

Try Visual Flow on AWS or GitHub

Apply for online demo

Visual Flow Logo

visual flow on AWS

visual flow on github

 

SPARKING ETL

A Low-Code Option for Building and Scheduling Spark Jobs.

We developed Visual Flow so you can jump-start Spark without writing a single line of code.

Until now, you’ve relied on a single-node ETL system. Now, your data is growing faster than your current engine can manage it. Distributed ETL engines like Spark offer tremendous upside. But when faced with coding jobs and pipelines manually, you run into a dilemma:

You don’t have time to learn coding languages
You can’t afford to hire external programmers
You don’t want to master an ETL tool, a build tool, and an orchestration tool
Your data continues to multiply out of control

The difficulty, expensiveness, and inefficiency of these options make Spark ETL seem impossible.

What You Will Get

By using Visual Flow, you will gain access to Spark ETL, transforming your data reserves into actionable wisdom.

Simplify ETL with a Drag-and-Drop Interface

Forget about Python, Scala, and Java, and embrace low- to no-code ETL on an easy-to-use interface.

Under-the-Hood Build and Orchestration

Visual Flow incorporates Argo Workflows, Kubernetes and the Spark processing engine. By mastering Visual Flow, you reap the benefits of all three software products.

Scale at the Speed of Data

Chain functions into jobs, then chain jobs into pipelines to transform raw data into Business Intelligence.

Visual Flow in Enterprise Infrastructure

How Visual Flow Works

Technical Features

Visual Flow is ETL, simplified. It allows users to leverage the scaling and efficiency capabilities of Spark ETL through an intuitive drag-and-drop interface.

Visual Flow includes:

  • Stage palette featuring nine discrete data transformation functions
  • Slack integration for user notifications
  • Support for data sources, including

● Relational DBs: IBM DB2, PostgreSQL, Oracle, MySQL, MSSQL, Amazon Redshift, ClickHouse

● NoSQL DBs: Elasticsearch, Cassandra, Redis, Mongo

● Cloud Object Storages: Amazon S3, IBM Cloud Object Storage

● Others: local files, dataframe

  • Ability to externally invoke Visual Flow pipelines using POST requests
  • GitHub and GitLab user directory support

Our Guiding Principles

Go Open Source

Visual Flow is an open source product that is based on other open source products.

Scale Optimally

Virtually unlimited horizontal scalability is provided by Spark.

Design Once — Run Anywhere

Once designed, deploy your ETL on any Kubernetes cluster anywhere.

Design Visually

Think of your data flows instead of the programming language syntax.

HOW TO USE VISUAL FLOW

Visual Flow is a low-cost, low-code, open-source product. Its portability, flexibility, and multi-cloud capability make it a powerful tool for a range of data projects.

  • Load Data from transactional sources into the data lake
  • Load application logs and structured files from a cloud object storage to the data lake
  • Cleanse and transform information inside the data lake
  • Load data from the data lake into the data warehouse

WHO VISUAL FLOW IS FOR

We developed Visual Flow for anyone looking for a convenient, intuitive ETL solution. Whether your data is multiplying faster than your single-server warehouse can process it, or you don’t have access to ETL coders for Spark, Visual Flow is for you.

  • Set up Visual Flow in under 30 minutes
  • Quickly design applications to transform large-scale, semi-structured datasets
  • Turn a messy backlog of data into uniform data lakes and warehouses
  • Intuitive, no-code interface
  • Standalone solution for intuitive ETL
  • Save money and time on training and/or new hires

Contact Us

    Yes
    YesPrivacy NoticeCookie Policy