Project topics and materials
Get Complete Project Material File(s) Now! »
GUI representation
Hayakawa et al. [2012] divided the user interface into four parts: Meta that contains information such as the title of the UI, Widget that includes the differents widgets types, Style that corresponds to CSS information, and Behavior that contains the code executed on user interactions. Following the proposition of other existing approaches [Fleurey et al., 2007, Garcés et al., 2017, Sánchez Ramón et al., 2014, Samir et al., 2007], we present all the visual aspect (Meta, Widget, Style) together in Section 2.1.1.
As depicted by Rodríguez-Echeverría et al. [2011] and Sánchez Ramón et al. [2016], the GUI layout must be represented in addition to the visual aspect. Thus, we present the layout representation in Section 2.1.2. Note that, to the best of our knowledge, there is no detailed representation of the Behavior part described by Hayakawa et al. [2012] in current published work. Thus, it will not be part of this state of the art and will be discussed in the following chapter.
Visual part meta-model
To represent applications GUIs, some authors propose to use meta-models. These meta-models represent the GUI in the context of the authors’ work. In the follow-ing, we discuss the proposed GUI meta-models.
OMG standards
The Object Management Group (OMG) defines two meta-models to represent the GUI visual aspect. The Knowledge Discovery Metamodel (KDM) allows one to represent any applications. The Interaction Flow Modeling Language (IFML) is specialized in applications with a GUI. The KDM standard defined a meta-model to represent a piece of software at a high level of abstraction. It includes a UI package representing the elements of a GUI.
Adapted GUI meta-models
Whereas the OMG proposed a standard to represent GUIs applications, several GUI meta-models have been proposed in the literature. As proposed by the OMG, all researchers used a hierarchical representation of the GUI in the form of a DOM [Gotti and Mbarki, 2016, Fleurey et al., 2007, Mesbah et al., 2012, Memon et al., 2003, Samir et al., 2007, Shah and Tilevich, 2011, Joorabchi and Mesbah, 2012, Brambilla and Fraternali, 2014]. Each node in the DOM tree represents a widget of the user interface. Thus, this representation is not controversial, and representing the DOM appears as a good solution to represent the GUI skeleton.
In addition to the DOM and the widgets, some authors added attributes. It is the case of Gotti and Mbarki [2016] who worked on analyzing the GUI structure of Java projects, Garcés et al. [2017] during the migration of oracle forms appli-cation, Memon et al. [2003] who extracted a GUI model from Java applications or MS Windows applications at runtime, Samir et al. [2007] during the migration process of Java application, Shah and Tilevich [2011] who extracted GUI to facili-tate desktop to mobile migration, and Joorabchi and Mesbah [2012] who analyzed IOS applications. In the light of the diversity of work, representing the attributes is essential when modeling GUIs. Others use the concept of events to later link the visual aspect with the GUI behavior. It is the case of Fleurey et al. [2007] for the migration of an industrial application, Mesbah et al. [2012] for the analysis of Ajax-based application and Garcés et al. [2017], Samir et al. [2007], and Joorabchi and Mesbah [2012] again. Representing events is a common task when migrating applications. In their work, the behavior of an application is only about navigation between pages.
Gotti and Mbarki [2016] and Sánchez Ramón et al. [2016] propose to specialize their meta-models with different kinds of known widgets such as Button, Label, Panel, etc. Thus, instead of using a generic widget concept, they can map each widget of the source code to its counterpart in their GUI meta-model. Finally, Sánchez Ramón et al. [2016] also define a special kind of widget named “custom” to support widgets that their GUI extraction approach might not recog-nize.
All researchers use the DOM to represent the visual part with the attribute and the event concepts. The authors also proposed modifications to adapt their meta-models to their specific context. However, no study presents how those meta-models can be used in other contexts. For instance, there is no proposed adaptation regarding the use of a meta-model representing desktop application GUI to repre-sent a web application GUI. Moreover, only Sánchez Ramón et al. [2016] present the concept of a custom widget to deal with unknown widgets. Thus, one needs to define an extensible GUI meta-model to reuse it in different contexts.
Layout meta-models
To correctly represent the visual aspect of a GUI, a layout representation is also necessary [Rodríguez-Echeverría et al., 2011, Sánchez Ramón et al., 2016]. There are three identified layout managers in the literature: hardcoded, hierarchical, and
constraint-based:
• Hardcoded layout defines for each widget its position with absolute coor-dinates on the screen [Sánchez Ramón et al., 2016]. It is used in old GUI frameworks. This layout is less and less used since the apparition of the DOM representation.
• Hierarchical layout consists of subdividing the available space of the screen into panels. Then the panels are responsible for placing their children in the dedicated space [Hasselknippe and Li, 2017]. Sánchez Ramón et al. [2014] propose a layout meta-model that supports hierarchical layouts. Zeidler et al. [2012] claim that the grid-bag layout, which is a hierarchical layout, is the most prominent and that almost all available GUI frameworks support it. Meliá et al. [2008] work on Model-Driven Development for GWT applica-tions. In GWT, a widget is at the same time a visual element and a layout. So, they designed a meta-model with GUI components in which the container el-ements are at the same time container and hierarchical layout manager, for example, the GridPanel, the HorizontalPanel, and the VerticalPanel.
• Constraint-based layout also uses a hierarchical structure, but it uses con-straints to place the widgets, for example: “place this button on the right of this text”. Lutteroth et al. [2008] presented the Auckland Layout Model, which is an implementation of a constraint-based layout.
Authors have proposed layout meta-models to represent GUIs accurately. Sev-eral representations exist depending on the research contexts. The hierarchical layout comes out as the most used layout manager.
GUI migration
Using their GUI representations, the authors have proposed migration approaches. We will first rapidly mention recent work on GUI generation using Artificial Intelligence (from screenshot examples). It is the case of Beltramelli [2017], Chen et al. [2018], and Moran et al. [2018]. These approaches rely on a huge dataset of screenshot examples (14,382 screenshots for Moran et al. [2018] and 10,804 for Chen et al. [2018]) to train the model. Thus, Beltramelli [2017] warns that the approach “is not, in any way, intended, nor able to generate code in a real-world context” and “both the source code and the datasets are provided to foster future research […] and are not designed for end-users”1. Consequently, we rule out We identified various publications related to migration. Section 2.2.1 presents generic approaches to migrate applications. Section 2.2.2 focuses on the language conversion research field. Section 2.2.3 presents existing projects migrating the visual aspect. Section 2.2.4 details the steps proposed in the literature to extract the GUI representation.
Table of contents :
1 Introduction
1.1 Context
1.2 Problem
1.3 Our Approach in a Nutshell
1.4 Contributions
1.5 Structure of the Thesis
1.6 List of Publications
2 State of the Art
2.1 GUI representation
2.2 GUI migration
2.3 Incremental migration
I GUI migration
3 Approach
3.1 GUI Terminology and Concept
3.2 Approach overview
3.3 Visual code migration approach
3.4 Behavioral code migration approach
3.5 Conclusion
4 Implementation
4.1 Visual code extraction
4.2 Visual code generation
4.3 Behavioral code extraction
4.4 Behavioral code generation
4.5 Conclusion
5 Migration Validation
5.1 Visual code migration validation
5.2 Behavioral code migration validation
5.3 Discussion
5.4 Threats to Validity
5.5 Conclusion
5.6 GUI migration conclusion
II Incremental migration approach
6 Incremental migration
6.1 Incremental Migration Approach
6.2 Hybrid architecture
6.3 Implementation
6.4 Conclusion
7 Incremental migration validation
7.1 Case Study: Omaje Application
7.2 Research Questions and Evaluation Methods
7.3 Evaluation Results
7.4 Incremental approach discussion
7.5 Threats to Validity
7.6 Incremental approach conclusion
8 Conclusion
8.1 Summary
8.2 Contributions
8.3 Future Work
Bibliography
