Project topics and materials
Get Complete Project Material File(s) Now! »
Software Development Process
A software development process defines the methods and procedures that organiza-tions and individuals have to follow to create software products and services [98]. Ultimately, the purpose of a software process is to provide a roadmap for the devel-opment of high-quality software products that meet the needs of their stakeholders within a balanced schedule and budget [137]. It focuses on the creation and mainte-nance of tasks rather than the output or the end product. Thus, a typical software process should aim to solve the potential and future problems of software develop-ment in terms of planning and budgeting. A broad definition of the software de-velopment process includes development, deployment and maintenance of a software product, which should include a set of policies, organizational structures (e.g., task definitions), human activities, technologies, and procedures [43]. Among the many software development approaches proposed in the literature, the most well-known ones are the traditional waterfall model and more recent agile approaches [92]. The four basic software development process activities of specification, development, validation and evolution are organized differently in different development processes [122] (see Fig. 2.1). In the waterfall model, they are sequentially organized, while in evolutionary development, such as agile methods, they are interleaved. The way in which these activities are carried out depends on the type of software, individuals and organizational structures involved.
1. Software specification or requirements management is the process of under-standing and defining the functional and non-functional requirements required for the system and determining the operational and developmental constraints of the system. The requirements engineering process produces a software re-quirements document that corresponds to the system specifications.
2. Software design and implementation is the process by which a system spec-ification is converted into an executable system by system design. A software design is a description of the architecture of the software to be implemented, the data that is part of the system, the interfaces between the system components and, sometimes, the algorithms used.
3. Software validation aims to demonstrate that a system meets its specifi-cations and customer expectations. It involves checking the processes at ev-ery stage of the software process. Most validation costs are incurred post-implementation when system operation is tested.
4. Software evolution, in particular software maintenance, is the term used in software engineering to refer to the process of developing software initially, then it is repeated updating for various reasons. The goal of software evolution would be to implement potential major changes to the system. The existing system is never complete and keeps evolving [75]. As it evolves, it becomes more and more complex. The primary goals of software evolution are to ensure system reliability and flexibility.
Release Management
Release Management is a fundamental approach for successful open source projects. Release management includes the planning, implementation and monitoring of soft-ware development processes. Release Managers are individuals who manage the software by administrating the release process, proposing release schedules, and en-suring that new releases progress timely [126]. A release is typically a named set of software components and supporting documentation that is subject to change man-agement and is upgraded, maintained and tested as a whole. A release has a release policy that defines the methods of how the release will be built, configured, and installed into the production environment [71]. In general, the release management process involves the following steps [15]:
– Requirements Gathering and Planning: When a new release is prepared, requirements and improvements needed to fix the current product are gathered.
The plan breaks the release into stages, sets up the overall workflow, and outlines who is responsible for each task.
– Release Building: The building process of the new release is started when the release requirements are identified and planned. It consists of the main activities used in the software development process.
– Acceptance Testing: The system release build is subjected to a comprehen-sive testing procedure when it is ready. The program is reviewed to verify if it works correctly and lives up to the requirements and dependencies.
– Release Preparation: The verified release is packaged and prepared as a final product to be delivered after checking if it meets the minimum standards of acceptance and business requirements as detailed in the release plan.
– Release Deployment: The release is deployed to the customer.
The system requirements evolve over time, so a release plan has to be defined that allows for changes and the gradual introduction of requirements. Release planning steers the project by defining milestones for when new releases have to be delivered. This forces the community to integrate their finalized code into the new releases. We present different types of releases, release channels and strategies to manage the release process in Section 2.2.1, Section 2.2.2 and Section 2.2.3, respectively.
Software Release Basics
In this section, we present the different release types delivered during the software development process. A build is an executable code that is handed over to the tester to test the functionality of the developed part of the project. Version is an instance of the software product that contains substantial and significant enhancements or other substantial change in functionality or performance compared to the previous version [45]. Whereas a release is a version of a software system that is tested and fully functional and has been made available to users for testing or public consumption [56]. Milestones are releases that include specific sets of functions and are released as soon as the functionality is complete [132]. Release engineering involves building fast and reliable pipelines to transform source code into viable products. Release engineering focuses on the purely technical aspects of the software release. In a nutshell, release engineering deals with the technical aspects of getting a release from development to production [112]. During software development, different types of releases are delivered [106]: There are two main types of release: internal/external testing releases and external product releases. For instance, the development organization may release a product internally to the testing group or may release a product to the user community for beta testing:
1. Internal and external testing releases: The software is tested to ensure that it works as intended before being available for public users. To this end, most organizations use alpha versions, beta versions and release candidates [35]:
(a) Alphas are used for internal acceptance testing where developers generally test the fix/change that has been produced.
(b) Betas can be considered as a form of external User Acceptance Testing, which is performed by users and communities. It starts when the software is declared to be “feature complete”, but may contain some known or unknown issues, such as performance or crashing issues.
(c) A Release candidate is a version of the software that is considered to be as the final software release. In this phase, no features are developed or enhanced; only issue fixes are allowed.
2. External product releases: Most organizations follow three levels of product
releases (semantic versioning): major releases, minor releases, and patches [102]. Major and minor releases are designed to be external product releases that are persistent and supported for a period of time.
(a) A major release represents the official version of a software product.
(b) A minor release is off-schedule and represents the same basic product but with enhanced features, performance, or quality.
(c) A patch release corresponds to a developmental configuration that is intended to be transient. It is produced to fix specific bugs or to add minor functionality.
Release Management Strategies
There are different strategies to manage the release process. Release strategies can be classified as feature-based, time-based or a hybrid of both [88]. Feature-based releases are delivered when a set of predefined goals has been achieved (e.g., when a set of features has been implemented or a set of reported bugs have been fixed). Time-based releases are delivered according to fixed time intervals, with whatever features are ready when the release date comes. A common type of time-based release strategy is the use of regular cycles where releases are delivered at regular intervals. The release cycle consists of all the stages from the start of the development of a new version of a product until its release to users, including the release of minor versions containing bug fixes or minor enhancements. When the release strategy is selected, the release cycle time is to be chosen. The release cycle time is defined as the time between the start of the development of a software product and its actual release [90]. Michlmayr et al. [90] identified five factors that affect the choice of release interval: regularity and predictability; nature of the project and its users; commercial factors; cost and effort; and network effects. A release model is a simplified description of a release process, i.e., of the activities carried out during the release cycle [123]. Table 2.1 highlights the aspects of two release models, traditional releases and rapid releases. Traditional releases follow a feature-based release strategy where every major release is delivered about one year or more after the previous release. Rapid releases follow a time-based release strategy where the time is reduced to a few weeks or months. There is a stabilization period in both release models, which occurs just before the release, it is called feature freeze. During this period, all work on adding new features is suspended, moving the effort to test and fix bugs to improve quality and stability. Under traditional releases, new features are implemented for the next release only. However, bug fixes are backported to older releases to guarantee that the old version users will get bug fixes in a timely manner [123]. Under rapid releases, it is usual to support only the latest version since releases are more frequent. The rapid release model allows the organization to deliver new features to users earlier. On the other hand, traditional releases may be appropriate when stability is more important than new features.
The ability to release rapidly was the second most mentioned benefit of agile software development, and it was practiced by 2 out 3 of respondents [16]. It was perceived that rapid releases enable easier progress tracking, easier monitoring of the quality of the software, more rapid feedback, a reduction of turnaround time and hence easier bug fixing. Agile software development highlights the importance of rapid releases, as one of its principles states “Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale” [14]. Rapid releases are claimed to offer a reduced time-to-market and faster user feedback [63]; end-users benefit from faster access to functionality improvements and security updates [80]; and improve turnaround time for fixing bad bugs [16]. Shorter time-to-market allows projects that adopt rapid releases can compete better with other projects and respond faster to customers’ changing needs [68]. Michlmayr and Fitzgerald [89] found that a reduced release cycle time can attract collaborators where the majority are volunteers motivated by the challenges associated with shorter release activities. However, Kerzazi and Khomh [60] stated that a release process needs to be automated and optimized in order to support rapid release cycles. Therefore, rapid release cycles are often adopted through modern release engineering practices such as Continuous Integration (CI), Continuous DElivery (CDE) and Continuous Deployment (CD) as they are an enabler for rapid releases:
Continuous Integration (CI) is a development practice that requires developers to integrate code into a shared repository several times a day. This practice aims to test the code frequently throughout the development process so that possible issues are identified and corrected early. Continuous Delivery (CDE) is an extension of CI where developed code is con-tinuously delivered as soon as it is considered ready for being shipped. It involves continuous integration, automation testing and deployment of automation processes, which allow rapid and reliable development and delivery of software with the least manual overhead.
Continuous Deployment (CD) is the next logical step after continuous delivery. It is the process of deploying the code directly to the production stage as soon as it is developed. In CD, all changes that pass the automated tests are automatically deployed to the production stage.
Table of contents :
Acknowledgements
Abstract
1 Introduction
1.1 Thesis Context
1.2 Thesis Statement
1.3 Contribution
1.4 Dissertation Structure
2 Background
2.1 Software Development Process
2.2 Release Management
2.2.1 Software Release Basics
2.2.2 Release Channels
2.2.3 Release Management Strategies
2.3 Bug Handling
2.3.1 Definitions
2.3.2 Bug Report
2.3.3 Bug Handling Process
2.3.4 Bugzilla Issue Tracking System
2.4 Summary
3 Related Work
3.1 Bug Handling Process
3.1.1 Bug triaging
3.1.2 Bug Resolution
3.2 Rapid Releases
3.2.1 Benefits and Challenges of Adopting Rapid Releases
3.2.2 Rapid Releases and Software Quality
3.3 Summary
4 Impact of Release Policies on Bug Handing Activity
4.1 Introduction
4.2 Methodology
4.2.1 Selected Case Study
4.2.2 Extracting and Processing Bug Report Data
4.2.3 Proposed Bug Handling Metrics
4.2.4 Applying the Metrics to Specific Eclipse Releases
4.3 Statistical Methods
4.4 Feedback from Eclipse Maintainers
4.5 Bug Handling Process Discovery
4.6 Applying Process Mining on Bug Handling Process
4.7 Quantitative Analysis of the Evolution of Eclipse Bug Handling Activity
4.8 Impact of Rapid Releases on the Bug Handling Process of Eclipse
4.8.1 RQ1.1 How does the bug handling rate evolve across releases?
4.8.2 RQ1.2 How does the bug handling time differ before and after each release?
4.9 Impact of Feature Freezes on Bug Handling in Eclipse
4.9.1 RQ2.1 How does the feature freeze period impact bug handling rate?
4.9.2 RQ2.2 How does the feature freeze period impact bug handling time?
4.10 Discussion
4.11 Threats to Validity
4.12 Conclusion
5 Revisiting the Impact of Rapid Releases on Software Development Activities
5.1 Introduction
5.2 Methodology
5.2.1 Selected Case Study: Mozilla Firefox
5.2.1.1 Mozilla’s Development Process
5.2.1.2 Firefox Testing & Quality Assurance
5.2.1.3 Mozilla’s Patch Uplifting Process
5.2.2 Data Processing
5.2.3 Statistical Methods
5.3 Impact of Rapid Releases on Quality Assurance
5.3.1 RQ1.1 : How does switching to more rapid releases impact the number of post-release bugs?
5.3.2 RQ1.2: How does switching to more rapid releases affect the number of manually performed tests?
5.3.3 RQ1.3: How does switching to more rapid releases affect the number of testers working on a project?
5.3.4 RQ1.4 : How does the frequency of intermittent test failures change after switching to more rapid releases?
5.4 Impact of Rapid Releases on Patch Uplifts
5.4.1 RQ2.1: How does the number of accepted and rejected uplifts evolve over time?
5.4.2 RQ2.2 : How effective are the patch uplifts and how does this change with more rapid releases?
5.5 Discussion
5.6 Threats to Validity
5.7 Conclusion
6 Conclusion
6.1 Contributions
6.2 Limitations
6.3 Possible Research Extensions
6.4 Future Work
A Online Form
Bibliography
