Operationalize Your Data

Run analytics at scale and realize the value of your data

Get Started with DataFoundry

Operationalize Your Data

DataFoundry greatly simplifies deployment and management of analytics use cases in production by automating:

  • Data Pipeline Deployment and Promotion – from development to production
  • Pipeline Orchestration – automated management of fault-tolerant analytic workflows
  • Hybrid and Multi-Cloud Deployment – automated export or migrate data pipelines to target platforms on-premises or in the cloud

Data Pipeline Deployment and Promotion

Capabilities

  • Easily export and import data pipelines from development to test to production with a few clicks
  • Integrate pipeline development and management with your CI/CD and SDLC processes
  • Abstracted source, target and infrastructure details from data pipeline logic

Benefits for Databricks users

Rapid promotion from development to production

  • Migrate configurations (not code) across different environments with built-in infrastructure and data connection abstraction
  • Ease the hand-off between developers and productions ops with code-less configurations

Automate your CI/CD processes

  •  Data pipeline logic is transported as configuration files and can be integrated with source code repos and processes

Pipeline Orchestration

Capabilities

  • Visual drag and drop designer for production workflows
  • Maintains audit trail of changes made
  • Run and monitor fault-tolerant production workloads with automated parallel execution
  • Centrally view and monitor execution times (current & historical), logs, errors and SLA
  • Automated retries with the ability to control retry and restart logic
  • Visually view process flow and 
the task descriptions
  • Dynamically control workflows with the use of parameters
  • Integrate external tasks including 3rd party logic
  • Directly access underlying processes from ingestion through transformations and export at any point in the process

Benefits for Databricks users

Orchestrate your complete data pipelines

  • Developers can easily orchestrate entire data pipelines, including external and off-cluster tasks, without needing separate production ops tools and expertise

Simplified monitoring and ops

  • Easily monitor production pipelines with full visibility into each task
  • Reduce troubleshooting efforts with easy restart and recover for fault tolerance

Hybrid and Multi-Cloud Deployment

Capabilities

  • Design once, deploy anywhere (on-premise, multi-cloud, multi-cloud)
  • Support for transient and burst workloads
  • Portability from one cloud environment to another with no rework required
  • Native optimization for maximum performance on each cloud compute engine (Databricks, HDInsight, EMR, Dataproc, Cloudera, etc)
  • Enterprise Cloud Bridge for on-premise to cloud data sync
  • Cloud replicator to rapidly replicate data across big data and object storage, with bi-directional incremental sync

Benefits for Databricks users

Flexibility

  • Retain the flexibility to migrate and run data pipelines in any environment whether on-premise, cloud or hybrid, without needing complex rework
  • Overcome data gravity by replicating data to your preferred compute environment
  • Access data whether it is on-premise or in the cloud, from where it need to be processed

Want to learn how Infoworks software automates data operations and orchestration for Databricks?

Contact Us