Introducing Cloud Data Integration Advanced Serverless
Serverless computing is a cloud computing execution model in which the application developers are freed up from managing and running the server infrastructure, virtual machines, or containers. Developers need only focus on building their application, thereby realizing higher productivity through operational simplicity. Serverless computing can be misleading as it does not mean that servers are not needed. In fact, servers are still used for running applications, but the responsibility of managing all the infrastructure and dynamically allocating machine resources is now with the provider rather than the application developer, so that you, the application developer, can focus on building the application logic.
A Paradigm Shift in Computing
We have seen a gradual evolution in computing in the last few decades. We have seen compute workloads transition from physical machines, to virtual machines, and then to cloud-hosted compute instances.
Lately, we have seen an adoption of container technologies whereby users were deploying and orchestrating their workloads using container technologies. Essentially, we are seeing a paradigm shift in the world of computing.
All these trends are in response to users’ needs to focus on building their core business applications and deliver business value rather than provisioning and operationalizing the infrastructure.
Serverless is the next evolution in this computing trend – a powerful paradigm that lets application developers focus on business logic rather than worry about scalability or server provisioning and maintenance.
Introducing Advanced Serverless
In our discussion with developers, we realized one of the biggest contributing factors for productivity losses in software development is the human cost of infrastructure setup and maintenance. Every hour spent provisioning machines, instantiating VMs, installing and patching updates is an hour lost in productivity.
With the goal of enabling developers to focus on business logic without worrying about infrastructure setup, we are launching Informatica Cloud Data Integration and Cloud Data Integration Elastic in Advanced Serverless deployment environments.
Advanced Serverless enables customers to leverage serverless computing to process data integration pipelines. Users can run either their data integration or their Spark-based data integration jobs in serverless mode. This eliminates the need to manage hardware or software and simplifies DevOps and DataOps, allowing developers to focus on business logic and deploy new data pipelines quickly.
Key Characteristics of Serverless Services
A good test of serverless is to ask of the service provider, if you will NOT have to:
- manage infrastructure at any point
- patch OS
- upgrade software
- manage scaling or elasticity needs of your job
- pre-provision infrastructure
- pay for idle infrastructure
Only if all these criteria are satisfied, only then can the service be called serverless! Informatica Advanced Serverless satisfies all these criteria and more.
Serverless is a powerful shift in how data integration and data engineering jobs are performed. However, there are other nuances like exception handling, deserialization, transformations, retrying, and monitoring that need to be implemented because they are not available out of the box. Informatica addressed all these needs through the Advanced Serverless offering. We do this by virtualizing the runtime environment, enabling developers to ingest, cleanse, and enrich data without ever having to think about servers.
Business Benefits of Advanced Serverless:
Lower TCO:
With Advanced Serverless deployment you do not have any servers to manage, and you do not incur any infrastructure maintenance costs. In addition, you do not need dedicated operational staff to manage the infrastructure. These translate to real savings, lowering your TCO.
Jump-start projects on day 1:
With many integration projects you have to start by planning for the deployment environment with questions such as: How many machines should we provision? What size servers and instance types are right for my needs? How much reserve capacity should I account for? How should I scale up and down my servers? And, what are the sizing and performance guidelines?
It could take many days or weeks to resolve and address these questions. There are too many decisions to make before even starting development. With Advanced Serverless, developers don’t have to worry about any of these issues, and they can jump-start integration projects faster. Developers are productive on day 1; they get to focus on developing their application logic instead of worrying about infrastructure needs.
Future proofing your integration needs:
Informatica Advanced Serverless decouples design time experience from runtime, thereby freeing developers from complexities of the runtime environment. Developers can develop integration mappings using the mapping designer and Informatica runs the mappings in our serverless cloud environment, including Spark processing. The decoupling between design time and runtime means, developers do not have to understand intricacies of Spark, or learn Java, Python, or Scala – they can continue using Informatica’s drag-and-drop interface, and we convert the application logic to instructions optimized to run on the underlying engine.
Higher productivity through operational simplicity:
With Informatica’s Advanced Serverless, developers and operations do not have to invest time in scaling and tuning their workflows. We take care of tuning the Spark parameters, Data Transformation Manager buffer size, and all the scaling needs through our machine learning-based, CLAIRE-powered auto scaling and auto tuning.
Faster time –to market:
Last but not the least, as developers start building mappings and working on their core business project, they are able to meet their business goals sooner. As a result, you achieve faster time to business goals and insights.
Key Capabilities of Informatica Advanced Serverless
Some of the capabilities worth highlighting for Informatica Advanced Serverless deployment include:
- Governance: automatically scales to accommodate your workload requirements for application resilience and for enduring unexpected spikes. This is important, but must be monitored carefully and you need to be proactive and configure monitoring alerts to avoid cost overruns. Informatica allows you to avoid such costly overruns by providing multiple governance guardrails such as:
- You can specify the max compute units to be allocated per job so that at any given point in time you can rest assured that the job will not incur more than the cost associated with the specified compute units for the job.
- You can also specify task timeouts to limit your cost exposure.
- Tenant Isolation: guarantees tenant isolation for customer workloads, as for every tenant we provision a DMZ that is exclusively reserved for running their workloads.
- Auto Upgrades: automatically upgrade the software for you without needing to plan and manage any upgrades.
- Auto-tuning and Auto-scaling: leverages the CLAIRE engine-based auto-tuning and auto-scaling to scale up and down the clusters and intelligently rebalance the workloads to maximize savings.
- Built-in High Availability: CDI Elastic Spark cluster provides automatic high availability and recovery for the Spark Kubernetes cluster.
- Security: built in multiple layers of security protections and controls from audit and threat protection to network security and access control policies to ensure compliance to industry standard security requirements. Customers have full control over the trusted network link that is used to establish secure communication with Informatica Advanced Serverless infrastructure.
Future-Proof with Serverless and Informatica
When deploying cloud applications, you should consider Advanced Serverless deployments first, and only consider the alternatives if serverless does not meet your demands. Advanced Serverless offers consumption-based pricing, auto-tuning and auto-scaling, and high availability, all without requiring a dedicated administrator to manage the environment.
Informatica is the Enterprise Cloud Data Management leader, and with the evolution of data management using Advanced Serverless, Informatica helps you future-proof your solutions using Informatica Intelligent Cloud Services Cloud Data Integration and Cloud Data Integration Elastic solutions. We have many more enhancements coming in Advanced Serverless. Please stay tuned for more blogs on this topic.
Next Steps
Learn more about Cloud Data Integration Advanced Serverless in this Meet the Experts webinar.
To find out if Informatica’s Cloud Data Integration Elastic service is right for you, try CDI-E free for 30 days.
