This is a sponsored post commissioned by Transwarp Technology Co. Ltd. Sponsor Posts are paid posts that appear on all pages of SiliconANGLE.com, supporting editorial efforts. For pricing and inquiries, email firstname.lastname@example.org.
Based in the coastal city of Yantai in China’s Shandong province, HengFeng Bank Co. Ltd. has grown quickly since its founding in 2003. It currently operates more than 210 branches throughout Shandong, Chongqing, Sichuan, East China, Fujian, Yunnan, Xi’an and Beijing and boasts more than 1 trillion yuan ($145 billion) in assets, up 70 percent in just two years. In 2015 it reported a net profit of 8 billion yuan on revenue of 23.8 billion yuan.
Born of the Internet age, HengFeng (previously called Evergrowing Bank Co. Ltd.) has made technology a cornerstone of its strategy. It was the first bank in China to use the cloud for its core banking system, and the first to build its analytics on the Hadoop big data platform. Its “Customer 360” initiative captures information about all customer interactions with the bank via all channels for use by account managers.
Information technology has been a double-edged sword for the consumer banking industry. On one hand, it has streamlined operations and introduced new channels to reach the customer. On the other, it has challenged banks to maintain the person-to-person connections that are so important to the banking experience, whether it’s routine transactions in a branch, small business loans or even commercial transactions. A recent McKinsey survey found that nearly 70 percent of Chinese consumers were open to working with a digital-only bank.
Big data presents an opportunity to re-establish connections with customers and strengthen brand equity based upon personalized products and services that appeal to each customer’s unique interests. HengFeng Bank is making this capability a cornerstone of its strategy.
“Technology has reduced the opportunities for banks to engage with their customers, so we are introducing digital channels to enable customers to bank with us at any time,” said HuiHui Li, CEO of the Consumer Bank at HengFeng in a 2015 interview with Fintech Innovation.
As HengFeng’s legacy data warehouse grew, it became clear that traditional, proprietary relational database management systems (RDBMS) would limit the bank’s ability to build a platform for the future. Such a platform would be too expensive to handle both the much larger data volumes required by traditional data warehouse applications and the need to support a new class of real-time applications.
Those real-time applications would facilitate capabilities like on-the-spot risk prediction and personalized product recommendations, whether through on-line channels or in person. HengFeng needed a unified platform to support both the data warehouse and real-time applications because all the functionality depended on a single, integrated method to access to all the data.
Open-source platforms built around Hadoop were a natural solution in terms of functionality and cost. But building a scalable warehouse that could also support sophisticated streaming capabilities would take time and require skills that are in short supply. The building blocks for advanced analytic applications that are part of the Apache and Hadoop ecosystems come with a high overhead of operational and developer complexity. That complexity is what finally brought HengFeng to Shanghai-based Transwarp Technology Co., Ltd.
Solution: Transwarp Data Hub (TDH)
Transwarp Technology released the first Hadoop distribution built on Spark in China (the Transwarp Data Hub) in 2013. Transwarp Data Hub features an analytic SQL database named Inceptor for the new data warehouse scenario; Discover, with capabilities for doing machine learning on big data; a NewSQL database, Hyperbase for handling large volumes of unstructured data with search functionality; and a SQL-based streaming engine for building real-time applications.
With the ability to support a scalable data warehouse and sophisticated streaming platform, TDH met all the criteria on HengFeng Bank’s checklist and was chosen as the bank’s big data platform.
Unlike the Hadoop distributions originating in the Apache Software Foundation and sold by U.S. vendors, Transwarp built much of the technology underlying Data Hub on a specialized version of Spark. While that required several years of effort, the new foundation yielded significant benefits in speed, scale and usability.
For example, developers can use a single language – the ANSI 2003 version of SQL – for batch processing, interactive analysis, graph analysis, streaming analytics and search. A wide variety of user-friendly business intelligence tools can work with TDH through ODBC/JDBC drivers, since TDH supports the full SQL 2003 standard. The analytic database, Inceptor, has more functionality than other Hadoop-based analytic engines. For example, Inceptor can analyze data from related tables in a single SQL query, even if some of the data resides in a remote RDBMS. Most Hadoop-based analytic engines can’t do that. Inceptor also supports vendor-specific dialects of SQL and stored procedures, which makes it easy for customers to migrate applications from competitive products to TDH. TDH supports 98 percent of Oracle’s PL/SQL syntax and 90 percent of IBM DB2 SQL/PL and Teradata SQL.
Stream, the streaming analytic product, leverages the very same SQL dialect for usability. Using SQL on streams simplifies analysis because developers can specify what they’re looking for exactly as if they were querying a DBMS. And because Stream supports the same SQL stored procedures as the DBMS, developers can build sophisticated applications that otherwise would require a separate language.
Stream can also process events both individually and in batches so that it can handle both real-time data as well as historical data, which greatly simplifies development compared to Apache Spark. Both Inceptor and Stream can also access the same machine-learning tools with SQL queries, which enables SQL to be used for much more sophisticated predictive analytics.
TDH’s speed and scale enables the same instance of the underlying platform to support not just the legacy applications but newer ones that require greater capacity and performance. Like other high-performance analytic DBMS, Inceptor supports an in-memory or SSD-based columnar store named Holodesk, which also provides index functionality for fast data access. The combination of a high-performance columnar store and the Transwarp proprietary Spark-based engine enables Inceptor to perform up to 10 times faster than open-source Apache Impala DBMS from Cloudera Inc..
Finally, Inceptor supports updates using ACID transactions. This enables analytics queries to run continuously while maintaining data integrity as new data is ingested. The result is that it’s possible to extend the amount of time available for analysis, since there is no need for extract/transform/load (ETL) procedures..
Outcome: Fundamentally New Applications
Migration and deployment were relatively quick and easy compared to what would traditionally be involved in a major platform change. The task of modeling data, optimizing stored procedures and rebuilding some indexes collectively took about two months. Once the migration was complete, HengFeng found the platform to be not only much less expensive to operate but also more scalable and easier to extend with additional services.
Cost savings have been significant. For example, the legacy hardware alone for the original Customer 360 application cost about $800,000. Those applications were re-deployed on commodity hardware for only $86,000 – a 90 percent cost savings. A $400,000 Oracle software license for Customer 360 was replaced with a $58,000 license for Transwarp’s solution.
The bank didn’t have to sacrifice performance for lower cost. Batch reporting based on the Oracle-based data warehouse used to take up to eight hours. That was cut to one hour with Transwarp. Integrating Customer 360 data was formerly a one-to-two-hour batch process. That was reduced to less than six minutes. A risk management application that used to require two hours to run now executes in 10 minutes.
The most crucial change was that the combination of an up-to-date platform and a fully integrated customer database greatly increased application flexibility. Both structured and unstructured information can now be combined. That flexibility also opens up new data sources, such as logs from the bank’s Web site, whether stored in a database or analyzed on-the-fly.
For example, the bank’s credit managers, investment advisers, and account managers all now have access to better information when they talk to clients. Account managers know about a customer’s complete activity across all channels and touch points, with real-time updates. That combination enables more informed and productive sales conversations.
New applications have also become possible. For example, China doesn’t have standardized credit ratings and histories. Instead, the bank integrates its internal data with data from the National Bureau of Statistics and other external sources of information on companies and individuals. When bank credit managers speak to clients about loans, they now have richer information with which to make risk assessments. The platform continues to monitor the information behind these assessments and alerts a manager if there’s abnormal activity that might change the risk.
Lower costs and a more capable processing engine also enable the bank to tackle new application areas that were prohibitively expensive or even impossible with its previous infrastructure. For example, a consumer-focused risk application now tracks multiple categories of previously dis-integrated data to enables account managers, credit managers and investment managers to provide intelligent recommendations in real time for products such as car loans, home refinancing or investments tailored to a consumer’s financial needs.
And the bank sees the potential for many years of innovation on the new platform. Digital banking in China is advancing at a rapid pace. Most payments are already done with cell phones using services from Alibaba Group Holding Ltd. and Tencent Inc.’s WeChat Wallet. HengFeng believes it has an opportunity over the next decade to restructure its operations and channels using mobile, cloud, real-time and big data technologies. Transwarp’s TDH will be the information infrastructure for this transformation.
Lessons learned: Bridge open-source and proprietary technologies
Proprietary data warehouses have delivered on their promise of supporting business intelligence on carefully curated data. But unified platforms that support much larger data volumes and data types, along with real-time analytics, are too expensive under traditional pricing structures. Open-source tools and ecosystems are considerably cheaper, but they also introduce administrative and developer complexity that can drive the hidden total cost of ownership to unsupportable levels. Open-source technology isn’t for everyone.
The answer for many customers is a spectrum of solution. Products like Transwarp’s TDH, Microsoft Azure and MapR Technology Inc.’s Converged Data Platform are examples of high-value proprietary functionality build on the foundation of open standards. Proprietary technology creates the uniformity that enables integrated platforms to be built. At the same time, customers can maintain standard interfaces and the cost benefits of open platforms.