Project topics and materials
Get Complete Project Material File(s) Now! »
DSU Scenarios and their Challenges
Updating an application without the need to restart it should guarantee that all the changes are performed and the state of the running application is not altered. In the scope of an object-oriented application, the state of the application is formed by the live instances in the environment. Guaranteeing the appli-cation state in OOP means that live instances should be preserved from one version to the other. Dynamically updating an application is performed in two different sce-narios: Live programming during Development (Development DSUs) and updating a long running application (Production DSU). Both kinds of DSUs share the challenges listed below. However, each of the families of DSUs present different approaches to solve these challenges. The different approaches focus on improving the results in one of the target scenarios.
• State Migration. Migrating the state of an application between versions is not a trivial activity. On the one hand, it requires a technique to re-place old values by new values (e.g., pointer swapping, lazy proxies). On the other hand, it requires a way to express value transformations which are usually application dependent and cannot be produced auto-matically. The migration of state is required to minimize the corruption of instances. A live instance is corrupted if after a change in its struc-ture or its usage it is not updated to follow the required new structure or usage [TPF+16].
• Change Identification. Determining the set of changes to apply (e.g., classes to create, methods to modify, instances to migrate) is error prone if done manually. Moreover, doing it automatically lacks precision: the process may miss business dependent value transformations required for state migration.
• Core Libraries and Self Update. Updating the core parts of a run-time environment (e.g., core language libraries) and the DSU itself is diffi-cult [PDF+15]. Such updates introduce circular dependencies that may break the update and require special mechanisms to ensure atomicity.
• Safe Point Detection. Detecting and deciding the best moment to execute an update (Quiescence Point, Safe Update Point or Alterability Point) [NH09] presents a challenge for those applications that were not designed for it. Looking for a safe update point should be fast enough to minimize the suspension time and smart enough to detect as soon as possible when such update point will never happen.
• Execution Penalty. Implementing all the above requires smart strate-gies to avoid performance penalties outside the update itself. For ex-ample, the usage of lazy proxies for state migration introduces an addi-tional level of indirection affecting the overall application performance.
DSU Practical Concerns
DSUs usage presents a set of concerns that should be addressed to have a practical solution. Performance. Making a program updateable should impact its performance as little as possible [HN05]. A DSU performance impact is divided in two stages: (1) during normal execution (outside update-window) and (2) during an update (inside update-window). A DSU should mini-mize the impact in both stages. Examples of impact are memory con-sumption, execution overhead and downtime during the update (Sec-tion 2.1.6). Ease of Use. The DSU engine should be easy to use by regular application developers. The less complicated the updating process is, the less error-prone it will tend to be [HN05]. A DSU solution should be integrated with the development tools used in the language. Also it should min-imize manual interactions and simplify them when they are unavoid-able. Finally, it should be present in the whole life-cycle of the applica-tion providing solutions during the development as well as during the evolution of systems in production (Section 2.1.7). Versatility. A DSU should be able to update any part of the running appli-cation [HN05]. The running application is not the only part that may require modifications. Core language libraries including the DSU en-gine itself also require updates (e.g., adding new features, improving performance or fixing bugs and security failures) (Section 2.1.8).
State Inconsistency
Applying updates to live programming environments change the structure and use of live instances. There are changes that leads to instance state in-consistency. Instance state inconsistency is when the internal state of live instances does not follow the expected value for live instances. This mis-match includes differences in existence or not (instance variables that ex-ists in only one of the version), how the instance variable is used (the same instance variable is used for different operations in different versions) and values types (the instance variable holds different types of values in differ-ent versions). These instances should be correctly migrated to avoid instance state inconsistency. A DSU in an object-oriented environment should manage arbitrary combinations of the following elementary changes. These changes are com-posed in different ways, allowing the creation of complex changes. Including changes in many classes at the same time, for example complex automatic refactorings. Adding new instance variable. Adding a new instance variable on an exist-ing instance means extending the structure of the object and filling the room with a value. Neither initializing the new variable with the null pointer nor using the value assigned in the construction are useful for existent objects. The decision of the value for the new instance variable depends on the up-date performed.
Removing instance variable. When removing an instance variable there are two options. Either the object is resized to fit its new layout and the value is dropped or the object keeps the value but the instance variable is made obsolete as it is not a property of its class any more.
Renaming instance variable. From the structure point of view this is equiv-alent to removing an instance variable and adding another one. But doing so, we loose the value held in the old instance variable. Therefore, the sys-tem needs to be able to transfer state from an instance variable about to be removed to a newly added instance variable.
Value Change. When the object structure is updated, it might not be a struc-tural change. Changes are application specific and only affect the value stored in an instance variable without modifying the structure of the object. These migrations may need to compute the new state of the object in various man-ners that involve all the data available in the object. The way the new values are calculated are application dependent. For example when in an object rep-resenting a product we store a price as a number. In a later version we store the price as an object (containing the number and the currency). The structure of the instances are not modified (both versions have the price variable), but the instances should be migrated and the migration is application-dependent i.e., what is the default currency.
Concurrency And Execution Inconsistency
During the update the application is execution, so it is possible that in the execution stack there are activations of methods that need to be updated. Also, the execution stack could include references to objects that should be migrated. The execution state of the application should be guarantee during and after the update. All the elements in the execution stack should be correctly updated or migrated. The DSU should execute the update only when it can guarantee that the execution stack of the application is not affected. Also, in multithreading ap-plications the updates and migrations should be synchronized.
From the point of the running application the update should be per-formed transparently and atomically.
Performance
Executing an update impact in the execution performance of the running ap-plication. A DSU should minimize the impact on the running application.
A DSU impact the performance of the running application if the applica-tion has an execution penalty because of the execution or presence of a DSU solution. There exist two different situations during the update window, and out-side the update window. On one hand, It is required that the DSUs do not affect the execution of the application outside the update window. On the other hand, the DSUs should minimize the length of the update window as the application is not executing during this window of time.
Ease of Use
If the DSU solution requires manual work from the developer, this should be minimized. Operation as the identification of changes, creation of a patch, expression of state migration logic and detection of safe point update should be per-formed automatically. In the scenario where the operations could not be completely automatic, the requirement of interaction from the developer should be minimal. Moreover, a DSU solution for live programming in development envi-ronments should give the ability to perform the changes in a iterative way. The developer should be able to perform the changes incrementally testing each of the modifications committing or discarding them without affecting the stability of the running application.
Finally the DSU should present a common usage interface and configu-ration for changes made in development and production. This transparency for the user improves the usability of the tool and the use during the whole development process.
Table of contents :
1 Introduction
1.1 Context
1.1.1 Live Programming Environments
1.1.2 DSU Solutions
1.1.3 Automatic Refactorings
1.1.4 Reflective Languages
1.1.5 DSU Scenarios and their Challenges
1.2 Problem Statement
1.3 Contributions
1.4 Thesis Outline
1.4.1 Part I: State of the Art
1.4.2 Part II: DSU for Production
1.4.3 Part III: DSU for Live Programming
1.4.4 Part V: Conclusion
I State of the Art
2 Comparing Existing Solutions
2.1 Requirements for DSU
2.1.1 Change Challenges Illustrated
2.1.2 DSU Practical Concerns
2.1.3 State Inconsistency
2.1.4 Change Interdependency
2.1.5 Concurrency And Execution Inconsistency
2.1.6 Performance
2.1.7 Ease of Use
2.1.8 Versatility
2.1.9 Requirements for a General DSU
2.2 Existing DSU Solutions
2.2.1 DUSC
2.2.2 Jvolve
2.2.3 DCEVM
2.2.4 DuSTM
2.2.5 JRebel
2.2.6 Javeleon
2.2.7 Javadaptor
2.2.8 Rubah
2.2.9 Pymoult
2.3 Categories of Existing Solutions
2.3.1 Classical Live Programming Environments
2.3.2 Production DSUs
2.3.3 Development DSUs
2.4 Related Techniques
2.4.1 Safe Point Detection
2.4.2 Migration Logic Generation
2.4.3 Benchmark and Validations
2.4.4 Architectural Solutions
2.4.5 Isolation and Atomicity
2.5 Analysis of Existing Solutions
II DSU for Production
3 Design Principles of gDSU
3.1 gDSU in a Nutshell
3.2 Patch Content
3.3 Patch Generation
3.4 Dynamic Patch Analysis
3.5 Thread Management and Safe Point Detection
3.6 Environment Copy
3.7 Application of Changes and Instance Migration
3.8 Validation and Commit of Changes
3.9 gDSU Platform Requirements
3.10 Conclusion
4 Designing Techniques for an efficient gDSU
4.1 Automatic Safe Update Point Detection
4.2 Efficient Partial Copy of the Original Environment
4.2.1 Detection of Modified Classes
4.2.2 Detection of Instances to Migrate
4.3 Reusable Instance State Migrations
4.4 Reusable Validations
4.5 Bulk Instance Replacement
4.6 Extensible Class Building Process
4.7 Conclusion
5 Validation of gDSU for production related requirements
5.1 Validation Set-up
5.2 Validation 1: Application Update
5.3 Validation 2: Update of the DSU
5.4 Validation 3: Update of Language Core Libraries
5.5 Validation 4: Benchmarks
5.6 Requirement Assessment
5.7 Conclusion
III DSU for Live Environments
6 Atomic State Preserving Refactorings
6.1 Class Refactorings that break Instances
6.1.1 Challenges in refactorings: Two examples of corrupting refactoring
6.1.2 Refactoring Impact Categories
6.1.3 Ubiquity of the problem
6.2 Our Solution: Atomic Refactorings for Live Programming
6.3 Preserving Instance State when Applying Refactorings with gDSU
6.3.1 Pull Up Instance Variable
6.3.2 Split Class Refactoring
6.4 Using gDSU to preserve instance state
6.5 Application of the Refactoring step by step
6.6 Validation
6.6.1 Validation 1: Refactorings without Corruption
6.6.2 Validation 2: Refactorings with Internal Corruption .
6.6.3 Validation 3: Refactorings with Complex Corruption .
6.7 Conclusion
7 State-aware Transactional Live Programming
7.1 Changes Corrupting Instances
7.2 Transactional Changes
7.3 Implementing PTm
7.3.1 Scoped Environment
7.3.2 Global State
7.3.3 State Conflicts Detection
7.3.4 Applying Changes
7.3.5 State-Migration
7.3.6 Aborting the Transaction
7.4 Using PTm to safely apply changes
7.4.1 Transactional Changes
7.4.2 Custom Migration
7.5 Transactional Modifications Validation
7.5.1 Validation 1: Manual Refactorings
7.5.2 Validation 2: Detection of Custom Migration Needing .
7.6 Design Decisions
7.7 Requirements Assessment
7.8 Conclusion
IV Conclusion
8 Conclusion
8.1 Contributions
8.1.1 gDSU and its techniques
8.1.2 Atomic Automatic Refactoring
8.1.3 State-Aware Transactional Live Programming
8.2 FutureWork
8.2.1 Distributed DSU
8.2.2 Isolation and Virtualization
8.2.3 Analysis of Changes
8.2.4 Language Evolution
8.2.5 Development Experience
Bibliography
