Improving Productivity, Quality and Security

Executive Summary

This white paper explores the transformative impact of generative AI (GenAI) on the software development life cycle (SDLC). The integration of GenAI is posited to significantly reduce manual efforts, cut development times, and enhance software quality and security by automating complex tasks and providing predictive insights. The key hypothesis is that GenAI can reduce manual efforts by up to 50%, significantly decrease development times, and improve the quality and security of software across its life cycle. The improvements vary by phase of the life cycle and an enterprise’s maturity level. In addition, as with any new technology, its application requires specialized knowledge and skills to avoid pitfalls that could impede the software development life cycle.

This paper combines the industry standard frameworks for Capability Maturity Model Integration (CMMI) and Common Weakness Enumeration (CWE) with multiple surveys and interviews conducted by ISG to assess the impact of GenAI on software development.

GenAI Impact on the Software Development Life Cycle

GenAI automation reduces the time spent on these tasks by 20-30% resulting in a 2-3% overall productivity gain in the Planning phase.

The software development life cycle is commonly described as consisting of six identifiable phases: Planning and Requirements, Design, Coding, Testing, Deployment, Maintenance. GenAI tools and techniques present a range of opportunities for impacting the work performed across these phases. Both the way in which GenAI can help and the magnitude of the impact will vary by phase.

During the planning and requirements phase, GenAI can automate stakeholder identification and requirement documentation through the use of AI-driven analytics and natural language processing. In the same way that GenAI is used to summarize other types of information, it can be used to summarize the information gathered from various stakeholders producing initial drafts of the documentation needed to move the project forward. This automation reduces the time spent on these tasks by 20-30% according to ISG research, resulting in a 2-3% overall productivity gain in this phase. Additionally, predictive models are used for feasibility analysis, further expediting decision making and enhancing efficiency.

The integration of GenAI significantly benefits the design phase in several ways. AI tools can automatically generate and evaluate architectural models, reducing design errors and time by 25-35% and resulting in a 3.75-5.25% overall productivity gain in this phase. AI-driven mock-ups create and test UI designs based on best UX practices, accelerating design iterations and enhancing user experience. This can cut down the time required for UI design by 20-35%. Additionally, GenAI optimizes database schema and data distribution, improving efficiency and scalability by 25%, which translates into a moderate productivity gain.

The coding phase sees some of the most substantial benefits from GenAI.

According to ISG research, the coding phase sees some of the most substantial benefits from GenAI. AI coding assistants can write and optimize code snippets, reducing coding effort by 30-50%, which results in a significant productivity gain of 10.5-17.5%. AI tools perform static code analysis, increasing code quality and reducing review time by 40%, contributing another major productivity gain. Moreover, predictive AI identifies and fixes common coding errors, decreasing debugging time by 45%, which significantly enhances code reliability.

Security Improvements From GenAI

Security is a critical aspect where GenAI delivers substantial benefits. ISG research shows that through automated code review, real-time scanning identifies security vulnerabilities early, resulting in a reduction of high-risk vulnerabilities and enhancing security compliance by 30-50%. AI-driven tools execute static and dynamic security tests, providing robust, real-time testing against known and emerging threats. With continuous monitoring and learning, AI detects vulnerabilities and applies security patches automatically, helping ensure long-term security through proactive maintenance.

A CWE vulnerability refers to the Common Weakness Enumeration, which is a category system for hardware and software weaknesses and vulnerabilities. It is part of a community project aimed at understanding flaws in software and hardware, and creating automated tools that can be used to identify, fix, and prevent those flaws. The CWE list is maintained by organizations like The MITRE Corporation and is sponsored by the U.S. Department of Homeland Security’s Cybersecurity and Infrastructure Security Agency (CISA). The CWE system includes over 600 categories of weaknesses, such as buffer overflows, path/directory traversal errors, race conditions, cross-site scripting, hard-coded passwords and insecure random numbers. These categories help developers and security professionals to pinpoint specific types of vulnerabilities within their software or hardware, allowing for more effective risk management and mitigation strategies.

The following table captures the transformative potential of GenAI in embedding security standards directly into the software development life cycle. By using AI, organizations can help ensure that code is not only functional but also secure by design, directly addressing the specific vulnerabilities as categorized by CWEs.

Testing Phase Optimization Using GenAI

Testing is another phase where GenAI can drastically improve efficiency. Test case design typically represents 25% of the testing effort. AI can generate and prioritize test cases automatically, reducing manual effort by 50% and increasing test coverage and relevance, leading to a major improvement in test efficiency. According to ISG research, test execution represents another 30% of the testing effort. Automated test execution including functional, integration and regression tests results in result significant productivity gains. Additionally, AI simulates user behavior and load scenarios, enabling accurate and efficient performance testing, thereby improving the efficiency of performance tests by 40%.

Automated assessments conduct comprehensive vulnerability scans, enhancing security by identifying and mitigating potential breaches more effectively. Continuous AI-driven testing automates regression testing, which although makes up only 5% of the overall testing effort, help ensure that new changes do not introduce bugs, maintaining stability and quality across multiple iterations with a 60% reduction in manual effort that contributes to overall testing efficiencies.

Enhanced Code Quality Through GenAI

GenAI enhances code quality through several means. Utilizing GenAI for code generation based on a corpus of best practices results in more consistent and improved code quality, particularly from junior programmers. Automating code reviews, detecting errors early and suggesting optimal coding practices further lead to cleaner and more reliable code.

ISG examined defect injection rates based on the CMMI framework. CMMI provides a structured, five-level approach to process improvement, helping ensure consistency, efficiency and quality in software projects. A 50,000-hour software development effort for a CMMI level 4 organization would result in 973 defects without GenAI compared with 681 defects if GenAI were applied. Additionally, integrating GenAI automates the generation of documentation and compliance reports based on CMMI standards, helping ensure up-to-date and compliant documentation, thereby reducing the burden on development teams.

GenAI significantly reduces defect injection rates across various programming languages. For example, it can reduce defects per line of code by 30%, resulting in cleaner and more reliable code. Focusing GenAI on more severe defects decreases not only the quantity but also the impact of defects.

Reduced Manual Efforts With GenAI

GenAI significantly reduces manual efforts across various sub-tasks in the SDLC. Automating routine activities, predicting outcomes and optimizing processes contribute to substantial productivity gains. Assisting with repetitive and mundane tasks such as documentation or resolving tickets improves efficiency and leads to greater employee retention. Reducing the resources required for these tasks enables enterprises to devote more resources to innovation and new capabilities, ultimately improving their competitiveness.

GenAI can assist with maintenance activities in several ways. By using AI-driven diagnostics tools to identify and resolve issues, enterprises can speed up the time to resolution. Employing AI to predict and prevent potential system failures reduces system downtime. Using AI for code reviews can also optimize system performance and enhance overall efficiency.

GenAI can also be utilized for documentation and compliance purposes in several ways. It can be employed to generate code documentation as well as user documentation automatically, helping ensure that the documentation remains up-to-date and compliant. Automating the generation of documentation reduces the burden on development teams. Additionally, GenAI can be utilized to translate documentation into multiple languages, meeting the needs of both development staff and users.

End-of-Life Opportunities

For software applications approaching end of life, GenAI offers the potential to relieve the financial and resource burden it can place on development organizations. Managing a software product or project is not just about launching the software; it also requires planning its entire life cycle through retirement or sunsetting. Ending support for a software product can negatively impact maintenance revenues and can damage an enterprise’s reputation, thus adversely affecting future revenue streams.

GenAI can help balance the tradeoff between costs and revenues generated at the end of a product life cycle.

GenAI can help balance the tradeoff between costs and revenues generated at the end of a product life cycle. Engaging a third-party resource to take over maintenance can free up resources to work on next-generation product development while also improving the bottom line. GenAI makes these types of options more realistic for all the reasons cited above. The ongoing maintenance, debugging and testing processes require less resources, enabling a software provider to continue to support the product for longer periods of time, more profitably. GenAI can also assist with the migration process to a third party by providing additional documentation and understanding of the code base.

GenAI Adds Value, but Requires Caution

Like most new and emerging technologies, GenAI offers many potential benefits, but also creates new challenges and risks. Enterprises need to be aware of these challenges and manage their use of the technology appropriately. When applied to the software development life cycle, GenAI can raise legal concerns regarding intellectual property issues. Determining the legal status of AI-generated code can be complex and contentious. Integrating AI-generated code with exiting code bases can be difficult, unless the new code conforms with the established coding standards and practices. Heavy reliance on GenAI for coding tasks might lead to a decline in developers' skills and expertise. This erosion of human capability can be detrimental in situations where human judgment and creativity are crucial.

Optimizing the testing phase with GenAI can offer significant benefits, but it also comes with several pitfalls that need careful consideration. GenAI can sometimes produce false positives, incorrectly identifying bugs, and false negatives, missing actual bugs. This can lead to wasted time and resources on non-existent issues or undetected vulnerabilities in the software. AI models are only as good as the data they are trained on. If the training data does not cover a broad range of scenarios, the AI might not be able to detect certain types of bugs or performance issues. This can result in an incomplete testing process where certain edge cases or rare bugs remain untested and undetected.

To mitigate these pitfalls, it is crucial to use GenAI as an augmentative tool rather than a replacement for human developers.

While GenAI can enhance code quality in various ways, it can also detract from code quality. GenAI models trained on specific datasets might be overfit to those data patterns and struggle to generalize across diverse coding environments and requirements. This can result in AI-generated code that works well in certain scenarios but fails to perform adequately in others, leading to inconsistencies and potential issues in production environments. AI-generated code might introduce new bugs or security vulnerabilities that were not present in the original codebase. The decision-making process of GenAI models can be opaque, making it difficult for developers to understand why certain coding decisions were made. This lack of transparency can hinder troubleshooting and maintenance efforts, as developers might struggle to interpret and modify AI-generated code effectively.

To mitigate these pitfalls, it is crucial to use GenAI as an augmentative tool rather than a replacement for human developers. Combining AI-driven code generation with rigorous testing, continuous human oversight and adherence to best practices will help improve the software development life cycle while addressing the inherent limitations of GenAI.

Case Study: Software Development With GenAI in the Telecommunications Industry

As the principles discussed in this white paper gain traction across the service provider landscape, ISG has observed significant productivity gains attributed to the adoption of GenAI in software development. Every major service provider interviewed in our research has accelerated software development with the integration of large language models (LLMs) at their core. Although it is still early to measure the full impact across the enterprise landscape, several compelling use cases are emerging, demonstrating the transformative potential of GenAI.

One of the largest telecommunications equipment companies in the world is a prime example of how GenAI is being leveraged to improve product lines and accelerate time to market. This industry leader has strategically adopted GenAI by establishing a Center of Excellence (CoE) with over 200 practitioners dedicated to integrating GenAI into their software development processes. This cross-functional team, consisting of both business and IT professionals, works closely to ensure the rapid and effective adoption of GenAI across the organization.

The company's CoE has focused on utilizing GenAI to enhance various phases of the SDLC, with notable successes in accelerating requirements gathering and code deployment. By integrating GenAI into their existing Integrated Development Environments (IDEs) coupled with AI-enabled repositories such as GitHub and GitLab, they have harnessed the power of AI to streamline coding, testing, and deployment processes. This integration has resulted in a significant increase in the speed and volume of code being deployed, which, in turn, has improved the overall efficiency and responsiveness of their development teams.

For instance, the CoE has reported a marked reduction in manual coding efforts, with GenAI automating routine coding tasks and assisting in more complex code generation. This has allowed their developers to focus on higher-value activities, such as innovation and strategic problem-solving, rather than routine maintenance tasks. Additionally, the implementation of AI-driven testing has enhanced the quality and security of the software being produced, reducing the time required for quality assurance and improving the reliability of the final products.

This telecom giant's journey to integrate GenAI-driven practices offers valuable insights for other enterprises looking to adopt similar strategies. The productivity gains and accelerated development timelines observed in this case highlight the potential benefits of integrating GenAI into the SDLC, particularly for large, complex organizations operating in fast-paced industries.

Conclusion

Integrating GenAI into the software development life cycle, particularly in design, coding, security and testing, significantly enhances efficiency, accuracy and quality. The impact of GenAI on the SDLC is profound, with potential productivity gains ranging from 20-50% across various phases. The efficiencies can also be applied to end-of-life scenarios to improve profitability and reallocate development resources. By reducing manual efforts, enhancing accuracy and improving security, GenAI offers a comprehensive strategy to modernizing software development. However, these improvements require caution and awareness of the risks of blindly applying GenAI without human oversight. But with the proper assistance, GenAI has benefits across all phases of the software development life cycle.