A Low-Code Option for Building and Scheduling Spark Jobs.
We developed Visual Flow so you can jump-start Spark without writing a single line of code.
Until now, you’ve relied on a single-node ETL system. Now, your data is growing faster than your current engine can manage it. Distributed ETL engines like Spark offer tremendous upside. But when faced with coding jobs and pipelines manually, you run into a dilemma:
You don’t have time to learn coding languages
You can’t afford to hire external programmers
You don’t want to master an ETL tool, a build tool, and an orchestration tool
Your data continues to multiply out of control
The difficulty, expensiveness, and inefficiency of these options make Spark ETL seem impossible.
What You Will Get
By using Visual Flow, you will gain access to Spark ETL, transforming your data reserves into actionable wisdom.
Simplify ETL with a Drag-and-Drop Interface
Forget about Python, Scala, and Java, and embrace low- to no-code ETL on an easy-to-use interface.
Under-the-Hood Build and Orchestration
Visual Flow incorporates Argo Workflows, Kubernetes and the Spark processing engine. By mastering Visual Flow, you reap the benefits of all three software products.
Scale at the Speed of Data
Chain functions into jobs, then chain jobs into pipelines to transform raw data into Business Intelligence.
Visual Flow in Enterprise Infrastructure
How Visual Flow Works
Visual Flow is ETL, simplified. It allows users to leverage the scaling and efficiency capabilities of Spark ETL through an intuitive drag-and-drop interface.
Visual Flow includes:
Stage palette featuring nine discrete data transformation functions
Slack integration for user notifications
Support for data sources including Amazon S3, IBM Cloud Object Storage, ElasticSearch and DB2
Ability to externally invoke Visual Flow pipelines using POST requests
GitHub and GitLab user directory support
Our Guiding Principles
Go Open Source
Visual Flow is an open source product that is based on other open source products.
Virtually unlimited horizontal scalability is provided by Spark.
Design Once — Run Anywhere
Once designed, deploy your ETL on any Kubernetes cluster anywhere.
Think of your data flows instead of the programming language syntax.
HOW TO USE VISUAL FLOW
Visual Flow is a low-cost, low-code, open-source product. Its portability, flexibility, and multi-cloud capability make it a powerful tool for a range of data projects.
Load Data from transactional sources into the data lake
Load application logs and structured files from a cloud object storage to the data lake
Cleanse and transform information inside the data lake
Load data from the data lake into the data warehouse
WHO VISUAL FLOW IS FOR
We developed Visual Flow for anyone looking for a convenient, intuitive ETL solution. Whether your data is multiplying faster than your single-server warehouse can process it, or you don’t have access to ETL coders for Spark, Visual Flow is for you.
Set up Visual Flow in under 30 minutes
Quickly design applications to transform large-scale, semi-structured datasets
Turn a messy backlog of data into uniform data lakes and warehouses