Pentaho - Lenient data conversion



         Usage of Lenient Data Conversion

Recently while upgrading Pentaho from older version 4.1 to 6.x I face some issues related to backward compatibility.I noticed that behavior of Pentaho 4.1 to 6.x is different while validating the data.

On digging the source code of PDI on git, I understood the behavior change made by Pentaho.

 The behavior till Pentaho 4.1 was not to change the source data as much as possible i.e. 
If you have some spread sheet which you mapped some column as integer and the value provided is $12.Pentaho will try to parse '$12' and extract 12 and store it!

While in Pentaho 6.0 (I have see the behavior in 6.0,it may have been introduced earlier) what is called as 'strict data conversion' has been implemented and made as default behavior.As a result of this Pentaho will no more try to parse above '$12' however will through data validation error.

For users, who don't want to change earlier behavior and want to live with it, can go for backward compatibility mode by passing following argument to jvm - 

-DKETTLE_LENIENT_STRING_TO_NUMBER_CONVERSION=Y


Comments

Post a Comment

Popular posts from this blog

How Istio service mesh can keep Spring Boot microservices light weight

How Istio service mesh can keep Spring Boot microservices light weight Over the years Spring Boot has become de facto chose for designing in building microservices. The micro service architecture brings lot of benefits by decomposing monolith into small independent components. However, which also brings common challenges related to distributed computing architecture. To overcome those challenges spring cloud has added many libraries like -         Eureka for service registry -         Hystrix for circuit breaker -         Zipkin for distributed tracing to name a few. All the above mentioned are cross cutting concerns and not limited to specific microservices. Hence one need to add these libraries into all micro services and embed them in the code by annotating them. This is invasive approach. As Kubernetes become preferred container orchestration platform, service...
Read more

My AWS study notes

Amazon Vitual Private Cloud (Amazon VPC) A virtual private cloud (VPC) is a virtual network dedicated to your AWS account. It is logically isolated from other virtual networks in the AWS Cloud. A VPC spans all the Availability Zones in the region. Amazon VPC consists of following components : Subnets  Route Tables Dynamic Host Configuration Protocol (DHCP) option sets Security groups Network Access Control Lists (ACLs) An Amazon VPC has the following optional components: Internet Gateways (IGWs) Elastic IP (EIP) addresses Elsatic Network Interfaces (ENIs) Endpoints Peering Network Address Translation (NATs) instances and NAT gateways Virutal Private Gateway (VPG), Customer Gateways (CGWs), and Virtual Private Networks (VPNs) 1. Subnets  A subnet is a segment of an Amazon VPC's IP address range where you can launch Amazon EC2 instances, Amazon Relational Database Service (Amazon RDS) databases, and other AWS resources. CIDR blocks define ...
Read more