AI Era Large Model Deployment Strategy in ToC Business Scenarios

2024-05-14 10:41:49

Financial institutions, in the course of business innovation and technology application, are subject to a variety of external factors, such as policy regulation and data privacy protection. With the continuous advancement of AI technology, how to promote the application of technologies such as large models in consumer business scenarios under these constraints, while making no

Financial institutions, in the course of business innovation and technology application, are subject to a variety of external factors, such as policy regulation and data privacy protection. With the continuous advancement of AI technology, how to promote the application of technologies such as large models in consumer business scenarios under these constraints, while making no compromises at the technical level, has become a challenging task.
At the 2024 Shenzhen ArchSummit Global Architect Summit, we will welcome a special keynote speaker—the technical lead from the user development department of Ping An One Wallet. He will share case studies on the application of technologies such as RAG vector search, knowledge bases, and annotation platforms in real business scenarios. He will also discuss the experience of initiating business projects, compliance regulatory approvals, and how to choose the right business lines from a technical and external practice perspective.
Prior to the summit, we had the privilege of conducting an in-depth interview with this tech expert. We look forward to him sharing his experience and insights on the practical application of large models in the finance industry, hoping this exchange will generate anticipation for his speech before the conference begins.

Question: As a technical leader, you have a very rich background in different areas of technology work, including client development, large-scale frontend, backend R&D and architecture management, and even recently entering the field of big data R&D. How have these cross-disciplinary experiences affected your technical vision and problem-solving abilities?

Answer: My technological journey began with Android development, where my in-depth work experience allowed me to fully understand the restoration and interaction logic of user interfaces (UI). This taught me to optimize user experience and response speed, and enhanced my understanding of underlying technological principles, such as memory management and performance optimization. In-depth consideration of the user’s perspective laid a solid foundation for my future career.

As the team grew and the technology stack integrated, my horizons expanded from an Android group to an app client team that integrated iOS development, and eventually grew into a large frontend team covering multiple technology stacks such as JS, React Native, and mini-programs. During this process, I learned about cross-platform and cross-browser compatibility issues, as well as methods for building efficient and maintainable frontend architectures.

Later, during the backend R&D stage, I gained a deep understanding of server architecture, database design and optimization, and key technologies for handling high concurrency. The company’s deep involvement in development gave me a deeper understanding of system stability and security. I learned how to ensure the accuracy and consistency of data in complex business scenarios.

Software development is an occupation that requires continuous learning. Experiences from various fields have come together to form a multi-dimensional way of thinking and provide a more comprehensive perspective when solving problems. As technical knowledge deepens and work experience accumulates, a developer can more comprehensively participate in the early stages of product planning, thus being able to provide guiding technical suggestions and professional technical support to business departments in terms of rapid product launch and saving on labor costs.

When it comes to the implementation of requirements, it is not just the technical aspects that require consideration, but also business and cost need to be taken into account. For different requirements, choosing the most appropriate technology stack is crucial. For example, in areas where regulation is strict and business updates are frequent, technologies that are easy to update such as React Native or H5 are more suitable. However, for environments that value user experience and have stable business operations, it is feasible to develop for both Android and iOS platforms. From a human resources perspective, implementing a requirement using front-end technology often only requires half the development effort of application ends, because each platform, Android and iOS, needs its own developer. This kind of thinking, rooted in technology yet able to transcend it, has facilitated my progress at work and brought significant benefits to the team.

Regarding the practical experience with technologies such as RAG vector retrieval, knowledge bases, and annotation platforms in the Ping An One Wallet app, the company and I have a strong interest in the field of large-scale AI models, and we have been seeking to apply them in company business scenarios. Last year, Ping An Group conducted an AI large model competition, evaluating multiple sectors including risk control, office work, and sales operations. The project for One Wallet was established based on communication and learning with many subsidiaries and teams. In the internet finance industry, One Wallet faces strict regulation. Constructing a comprehensive product faces challenges in data privacy and legal compliance, and RAG+ knowledge base is perfectly suited to this business scenario. It does not only leverage the capabilities of large models but also ensures the boundaries of input and output, thus effectively managing data privacy and compliance.

At the enterprise WeChat end, we have an abundance of customer resources, but the response from operational personnel is limited, making it difficult to fully utilize marketing potential. Initially, we selected a business scenario for marketing, trained the corresponding knowledge base, and put it into use, achieving positive results. Many business departments are interested in utilizing RAG technology and wish to apply it to their own operations. To address the issue of the need to isolate different business knowledge bases and annotations, we established an annotation platform. This platform primarily serves business departments, solving issues such as test case libraries, prompt word adjustment, and model regression, accelerating the iteration of projects, and improving accuracy. Additionally, this project has trained a large number of annotators for the company, which is an extra achievement.

When dealing with private domain large model technology ToC-end full-process architecture, not all challenges encountered are related to technology. The initial difficulty was in selecting business scenarios and obtaining support from the business side. The promotion did not succeed in securing resources from the business side, and without their participation, technology cannot operate in isolation.

Our team transformed the competition project into a comprehensive corporate knowledge base, encompassing company culture, onboarding training, contract terms, and other aspects, encouraging all employees to participate in the experience. This interactive experience, coupled with our promotion efforts, gradually won the support of the first business team. However, during the promotion, we faced legal and regulatory challenges. The state has a set of reporting and approval procedures for consumer-oriented large data models, which is a new field for the entire group. To meet this challenge, we took the lead in adopting technical solutions, preparing in areas such as data protection and refusal databases, while arranging personnel to proactively communicate with regulatory departments and actively provide required materials.

About the data security and privacy protection of private deployment solutions, we have adopted multi-level technical measures: access rights management, data encryption, physical isolation, data desensitization and anonymization, differential privacy, homomorphic encryption, and intranet deployment, etc. It is particularly worth mentioning that the core tenet of private deployment is to ensure the security and privacy of data. Even during the data transmission process, encrypted channels are established to ensure the security and privacy of the data. Technological development often precedes regulation, and it is usually the emergence of new technologies that renders existing rules inapplicable, thereby prompting the introduction of new policies. Therefore, if one wishes to realize technological dreams, one must maintain a leading position. Before the popularization of similar products, we need to maintain a spirit of pioneering and innovation, dare to try unknown things, and continuously learn cutting-edge technologies to successfully implement projects and achieve technological innovation.

In the AI tool ecosystem, annotation platforms play a crucial role. Developers cannot complete RAG projects independently; they need front-line personnel who have a deep understanding of the business to participate in the construction of the knowledge base and the annotation work. In the process of establishing the annotation platform, we involve business teams to join in the adjustment of new prompts, updates to the knowledge base, model validation, and the enrichment and perfection of the test case library, thereby improving the stability of model results and the speed of project iteration. Through this method, we cultivate more talent who understand big model annotation within the company and also accelerate the pace at which different businesses go online. Initially, the main challenge we faced was the high learning cost for business personnel on the annotation platform, which we addressed by producing documents and guidance manuals to reduce this cost, and the training period for new members has now been significantly shortened.

Lastly, let’s talk about the balance between technological innovation and regulatory oversight in the financial and banking sectors. Regulatory policies are the red lines of business operations that must not be crossed. In response to this, our strategy is to first deeply understand the relevant laws and regulations, and assess the feasibility of the solution. Next, we consult the company’s legal department for their opinion, and as long as there is no explicit prohibition, we begin to explore possible solutions. In addition, drawing on successful domestic and international cases and experiences is also very important. If there are no ready-made cases, we seek platforms for discussing cutting-edge technologies, such as closed-door roundtable meetings and industry conferences, to discuss and solve the challenges encountered.

Achieving a positive interaction between technological innovation and regulatory policy is a complex and difficult challenge. Reaching this balance effectively requires that talents possess both professional legal knowledge and cutting-edge technical capabilities, and such talents are extremely rare. In reality, technological innovation often leads regulatory policy, as is the case with ride-sharing services, drones, and big model technologies, among others. As technical professionals, it is advised that we look to the forefront of technology and continuously enhance our professional capabilities to better understand and assess the risks and benefits brought by technological innovation, and provide legal staff with judgment basis to assist in finding the proper balance.

When it comes to the application of private domain big model technology, the decision-making of business lines needs to be carefully considered. Under ideal support conditions, the implementation of a project should take into account multiple factors such as the strategic goals of the enterprise, business needs, technical feasibility, risk control, and regulatory requirements. For different business lines’ needs in strategic alignment capabilities, it is first ensured that the project is consistent with the overall goals of the company; secondly, business needs are assessed to ensure that the investment-return ratio is valuable; then, the feasibility of technology deployment is evaluated, including technical aspects and regulatory compliance. For those demands considered valuable, their priority will be elevated to provide support. If the technical team does not have the decision-making power and needs to seek support from business departments, the strategy needs to be flexible: place themselves in a sales role, consider the private domain big model as a product that needs to be promoted, actively explore the pain points of different business sides, and provide them with big model solutions, thereby persuading them.

In the process of exploring the application of proprietary large-model techniques, sensitive data in the internet financial payment industry requires extra attention. This data includes highly sensitive information such as user transaction information, personal identity data, and payment habits, which demands very strict protection measures. Proprietary large-model technology can analyze and predict data through localized deployment of large models while ensuring data security and privacy, thus avoiding direct data transmission to third-party platforms and reducing the risk of data leakage. Considering the high requirements for response speed and system stability in payment scenarios, proprietary large-model technology tends toward lightweight models and edge computing. By utilizing model compression, knowledge distillation, and other techniques to simplify the model structure and reduce the demand for computing resources, complex large models can run efficiently on mobile or terminal devices, which not only reduces latency, improves service quality but further reduces security risks during data transmission.

With the continuous maturation of technology, it’s anticipated that large models will achieve a deeper integration and intelligent whole-link in the internet financial payment industry. From user registration, identity verification, transaction authorization, anti-fraud monitoring, credit approval, customer service to market marketing and other aspects, large-model technology will serve as a core driving force, providing intelligent solutions and comprehensively enhancing the efficiency and security of payment processes.

The conference agenda has been expanded to include a series of special topics and in-depth discussions, covering the latest developments and industry insights in areas such as large-model applications, architecture upgrades, smart computing platforms, AI programming, and cost optimization.

The conference has now entered the countdown stage for discounted ticket sales. Seize this opportunity and you will be able to enjoy discounted ticket prices. Please contact the ticket manager in time to reserve your seat and enjoy the latest discounts.