Abstract

 

The point of this project was to produce a Portable Java IDE, which allows students/programmer to be able to do there work anywhere. The idea is that users can run the IDE from a website via applet and JNLP and connect to a server which contains where work. The IDE contains the most common features required my programmers. The IDE has the following functions


Acknowledgments

 

 

I would like to thank Prof Uday Reddy for helping me on this project, and giving me useful advice. I would like to also thank the third year computer science students who took some of the time out to answer some of my questions and to give useful criticisms on my IDE.

 

Finally I would like to thank all those developers who produced open source libraries which were vital in order for the IDE to be complete (e.g. ANLTR and Apache ANT).


Table of Contents

1.0 Introduction. 3

1.1 Personal Views of what makes a good IDE?. 3

1.2 Other Students Thoughts. 4

1.3 What my programme will contain and do. 4

2.0 Project Management 6

2.1 Overview.. 6

2.1 Choosing a software development methodology/model 6

2.2 Basic Feasibility Study. 6

2.3 Swot Analysis. 7

2.4 Risk Analysis And Management 7

2.4.1 Identification. 7

2.4.2 Time / Project Schedule Over Running. 8

2.4.3 Performance. 8

2.4.4 External Components. 8

3.0 Requirement Specification. 9

3.1 User Requirements Definition. 9

3.2 Non-functional Requirements. 10

3.3 Efficiency. 10

3.4 Portability. 10

3.5 Prerequisite. 11

3.5.1 Software. 11

3.5.1 Hardware. 11

3.6 Attributes. 11

4.0 Design. 12

4.1 Overview.. 12

4.2 Simple Design. 12

4.3 Auto completion Design. 15

5.0 GUI Module. 17

5.1 Overview.. 17

5.2 Feasibility. 17

5.3 Implementation. 19

5.3 Testing. 20

6.0 Java Editor Module. 21

6.1 Overview.. 21

6.2 Feasibility. 21

6.3 Implementation. 21

6.3 Testing. 23

7.0 Auto Completion Module. 24

7.1 Overview.. 24

7.2 Feasibility. 24

5.3 Design. 26

7.4 Java Auto Completion. 30

7.5 Implementation. 32

7.5.1 API Reader Module/JavaPackages. 32

7.5.1.2 Performance. 32

7.5.1.3 Testing. 33

7.5.2 Trie Module/Data Structure. 33

7.5.3 Testing. 35

7.5.4 Parser Module. 35

7.5.4.1 Incremental Parsing. 35

7.5.5 AST Analyser Module. 37

7.5.6 Auto Completion Analyser Module. 37

7.5.7 Performance. 38

8.0 Compiler/Runner interface module. 39

8.1 Overview.. 39

8.2 Feasibility. 39

8.3 Implementation. 40

8.4 Testing. 40

9.0 External Plug-ins interface module. 41

10.0 JavaDoc interface module. 42

10.1 Overview.. 42

10.2 Feasibility. 42

11.0 Server/Client module. 43

11.1 Overview.. 43

11.2 Feasibility. 43

11.3 Implementation. 45

11.4 Testing. 47

11.5 Security and Reliability. 48

11.6 Portability. 49

11.7 Performance. 50

11.8 Integration & User Interface. 50

12.0 Testing. 52

12.1 Automated Testing - JUnit S1 52

12.2 Logging - Log4J S7 52

12.3 Visual Logging - Graphical Components in Log4J. 53

12.4 Simulator 54

12.5 Using my IDE to write my IDE. 54

12.6 Computer Science Students. 54

13.0 Analysis. 56

13.1 How the IDE meet its requirements. 56

13.2 GUI and Usability. 56

13.2 Reliability. 57

13.3 Performance. 57

13.4 Portability. 59

13.5 Cost 59

13.5.1 Personal Cost 59

14 Conclusion. 60

14.1 Possible Extensions. 60

14.2 Lessons Learnt 62

External Software/Tools used in Project 63

Bibliography. 64

Appendix. 65

Appendix A- Architectural Design. 65

Appendix B- Initial Prototype. 66

Appendix C - Generic Build File for Apache ANT. 67

 


Table of Figures

 

Figure 1 showing the auto-completion facility in JBuilder, Forte and IntelliJ IDEA.. 10

Figure 2 showing the waterfall model 11

Figure 4 shows the external interactions of my project 17

Figure 5 shows the simplified and incomplete architecture for the IDE. 18

Figure 6 shows the architecture of the auto completion facility. 21

Figure 7 shows the main GUI for my IDE. 22

Figure 8 shows a screen shot of the create new java file dialog. 23

Figure 9 shows a screen shot of the import files dialog. 24

Figure 10 shows class diagram of the GUI module. 25

Figure 11 shows a screenshot of the start-up screen. 25

Figure 12 shows the class diagram for the editor module. 27

Figure 13 showing an example of some java code. 29

Figure 14 shows the steps involved in compiling source code. 30

Figure 15 shows the individual steps to perform auto completion for method/field names. 30

Figure 16 showing an example of a complete java code. 31

Figure 17 some additional text being added to Figure 16. 31

Figure 18 showing an example of a trie data structure. 31

Figure 19 showing simplified diagram of the classes involved in auto completion. 32

Figure 20 shows an alternative view to the diagram in Figure 19. 32

Figure 21 showing (UML) class diagram for auto completion. 34

Figure 22 showing the class diagram of the java auto completion module. 36

Figure 24 showing class diagram of the Trie module. 39

Figure 26 shows some java code and the corresponding AST. 40

Figure 27 shows the java code in Figure 26 being updated. 41

Figure 28 shows my compiler module compiling some code, which has error in it. 44

Figure 29 is a class diagram showing what classes are used to interact with any external software/plug-ins  46

Figure 30 shows a screen shot of the javadoc module. 47

Figure 31 shows a table containing the explanation of each header/key that is used in my protocol 50

Figure 32 shows a class diagram for the client/server module. 51

Figure 33 showing a screen shot of two clients connect to the same server 52

Figure 34 showing the login dialog for SSH connection. 53

Figure 35 shows a screenshot of files being locally cached onto clients machine. 56

Figure 36 showing a dialog for connecting to a server 56

Figure 37 showing a screen shot of instant messaging. 56

Figure 38 shows a screen shot of the types of messages what the server sends and receives. 56

Figure 39 shows JUnit running a test case. 57

Figure 40 shows a screen shot of logging in my IDE. 58

Figure 41 shows a screenshot of LumberMill that has been modified to interprete components. 59

Figure 42 shows a table giving a critic on how each section met its requirement 61

Figure 43 showing table comparing the size of different text editors. 64

Figure 44 shows a table comparing my IDE with XEmacs, JBuilder and Netbeans. 67

Figure 45 is a screen shot of my proto-type. It showa how the java text is broken down into a tree. 73

 


1.0 Introduction

 

This report discusses a third year software project, and documents the specification, design, construction and testing of the software.

 

I have been at Birmingham University for a couple of years now and have not been impressed with the current tools used to write java programmes. The main tool in question is a programme called XEmacs. XEmacs provides syntax highlighting, and ability to compile/run single class files.

 

I however believe that this provides the most basic facilities for writing java programmes. So my project it so produce an IDE/editor to help students like myself to write java programmes easier.

 

1.1 Personal Views of what makes a good IDE?

 

After using programmes like Forte and Jbuilder, I found that these IDEs makes writing java programmes much easier. The main tool which made the biggest different was auto completion facility (refer to Figure 1 for demonstration). This is where the user can type a name of a variable/object and all the method/field names for that object are shown, and the user just selects which one he/she wants.

 

This has many advantages

·         Firstly, it reduces syntax mistakes as it types the method name for you. (The most common mistake that I and I would imagine most students would have in XEmacs would be misspelling of method/field names.)

·         Secondly and most importantly it gives you a very brief overview of the API. As the method names itself provides enough information to tell the programmer what the method does. As a result to do not need to refer to the java API.

 

That single facility can save a lot of time, but most importantly the programmer spends more time on the semantics of the java programme instead of the syntax.

 

Also from the second year on wards, we have started to use packages in java, and XEmacs does not give any support for packages. As a result the programmer must use command-line to compile and run their files.

 

The problem with both Forte and Jbuilder is that it requires high spec machines, and for the novice user, it can seem somewhat over whelming with all the additional options. This is probably why the university does not provide this type of software.

 

Another problem that arises at university is the tedious task of updating/transferring my java code. This comes about as I don’t use one single computer to do my work on, but instead they use computers both at home and at university, with multiple operating systems, which all have the own file systems.

 

A possible solution to this problem is to also integrate client which can connect to a server that’s contains my work. Once this is done it also gives the ability for multiple users to work on the same java project simultaneously. (This is useful for team projects).

 

However to take it one step further and produce a truly portable IDE, in which I (or any other student) can do my work anywhere (home, university, cyber café, and so on) can be achieved as long as I can ensure that my software can run on java runtime 1.4 and from a website. (They can run my IDE via an applet/JNLP).

 

XEmacs, Forte, JBuilder and other IDEs all require installation and a JDK (java development kit). My aim is to remove these constraints and to produce a more flexible/portable IDE.

 

1.2 Other Students Thoughts

 

I have so far raised my own person thoughts of what makes a good java IDE, however to get a much more accurate idea. I decided to ask fellow third year computer science students (20) what their thoughts were on XEmacs, and what improvements could be made.

 

The below bullet points show what were the main difficulties that students had about XEmacs, and generally programming in university. (Bullet points are ordered from most common answers)

L                  Poor user interface

L                  Doesn’t compile all your java code.

L                  Strange characters appear in the code (caused by saving work in a windows platform which uses carriage return and new line for every enter button)

L                  Problems of copying work onto university computers

L                  Can’t run XEmacs on home machine.

L                  Doesn’t include any external libraries when compiling and running

L                  Poor printing support

 

The question regarding what improvements could be made where

J                  Better user interface

J                  Better compiler

J                  GUI builder

J                  Auto completion

J                  Producing UML diagrams of code

J                  Being able to connect to university and continue doing work.

J                  Automatically underlining errors in java code (similar to Microsoft Word when there is a spelling mistake)

J                  Running XEmacs on any platform

J                  Having a java debugger

J                  Including refractoring support

J                  Software that checks the style of your code. (Is your code designed properly, do all the classes have a capital letter and all variable/method names start with a lower case character)

J                  Better printing support

J                  Better javadoc support

J                  Have better support for other languages like Haskell, Prolog, C++.

J                  Group support (allows a group of users to work on project simultaneously)

 

As you can see students had a wide range of ideas regarding improvements. However the problem with this project it doesn’t require one single feature to fix the problem, but instead required a huge number of mini features to achieve the aim.

 

I am only one person working on this problem, I will not be able to include all these improvements instead I will attempt to choose the main points that the students raised (and my own person points) and design my software so it can be easily extended.

1.3 What my programme will contain and do

 

·         Produce a basic text editor with syntax highlighting. (Include cut, copy, paste, undo, redo facility, and basic code beautifier)

·         Introduce the auto-completion facility (in particular object/variable completion, refer to Figure 1)

·         Ability to compile and run java class files.

·         Ability to produce and read javadoc

·         Flexible look and feel

·         Portable IDE or implement ability to do work anywhere (i.e. allow remote access to work by using Client/Server architecture)

·         Implement group support

 

The reason why I have decided to do the auto-completion is that it requires me to analyse the java code by producing an abstract syntax tree, and for some other features that students wanted such as GUI builder, refractoring would also require this. Fundamentally I am trying to produce and IDE, which contains very useful features to a java programmer, but also produce a good framework to allow the IDE to be easily extended.

 

The reason that I decided to implement group support even though it was one of the least requested features is due to the fact it will only require slightly more work when designing the remote access feature.

 

Figure 1 showing the auto-completion facility in JBuilder, Forte and IntelliJ IDEA

               

 

2.0 Project Management

2.1 Overview

 

Project management is important in any software project, which isn’t trivial. It keeps tabs on how the project is going, where it should be and chooses the best course of action. As result prevents the project running late or at least be aware of it much earlier on in time.

2.1 Choosing a software development methodology/model

 

As you already seen my project will consist of many mini features which are distinct between each other. These features can be split up into modules. Therefore, its best to produces these features independently to each other.  Instead of typing all of them in one go, i.e. using the waterfall model.

 

The water model can be seen in Figure 2, and as you can see it there are distinct stages, design, implementation and testing.

 

Idea of this is quite good if you are handling only one programming function/module. However this is not the case for me instead I’ll be using an incremental software methodology.

 

This is very similar to water fall model in which there are distinct phases, however this is incrementally done on each feature/module.

 

Figure 2 showing the waterfall model

However in order for this to work you’ll need an initial basic design, which can be seen in design chapter on page 13.

2.2 Basic Feasibility Study

 

Is the software even feasible? Yes, as there are programmes like JBuilder and Forte, which have the some of options, which I intend to integrate.

 

Is it technically feasible in java? Yes, as JBuilder and Forte are written in java, however they may contain native coding.

 

Is it technically feasible for me? Auto completion is very difficult thing to do. As it requires analysis of java code. All other components/features are technically feasible for me as I have some prior knowledge, which is essential to this project.

 

Is it possible to finish project on time? This is a much more difficult question to answer. The reason being is that I am attempting to implement a lot of features. The only way I can ensure that I can finish it to use external components to help me achieve some of the features and for the rest I’ll reduce the scope of each feature, but ensure that it can easily be extended.

 

Can I plan the project? Yes especially, if you are referring to top-level planning, i.e. the core modules/milestones then yes very easily.  However the finding out the exact time to complete each milestone is going to more difficult. Therefore, best thing to do is to start on the project in the summer for contingency, in case my estimates are inaccurate.

 

Can I meet the non-functional requirements such as performance or user interface? For the user interface, I let fellow third years students try out my IDE and get feedback of what they thought of the GUI. On the performance front, I will need to identify exactly which modules have a performance constraint, then do a much more detailed feasibility study on those modules.

 

Which programming language should I write my software in, or should I merely extend XEmacs? Firstly I will start on the reason why I can’t use XEmacs. As you read in ‘1.2 Other Students Thoughts’ XEmacs has many flaws, such a poor user interface, but most importantly my project is to produce a portable IDE, XEmacs is not portable.

 

On choosing which language to write my project in, will be without a doubt java, due to the following reasons.

ü      Auto completion will require me to find out the method/field name of each class/object, this can only be achieved by interacting the java runtime.

ü      Java is not platform dependent therefore it is portable.

ü      Java will allow me to up put programme onto the web via applet/JNLP.

ü      Also I already know how to programme in java

2.3 Swot Analysis

 

Strengths

·         Started project early (in summer)

·         Good at programming in java

·         Project is flexible

·         I am good at doing research on a topic

Weaknesses

 

·         Have very little experience in optimising code

·         Can only programme in java.

Opportunities

 

·         New API’s/components/technologies come out which can help me on the project

·         I am doing compilers and languages module, which could help me for the auto completion section.

Threats

 

·         Cannot finish project on time.

·         Cannot meet the performance requirement

·         External tools do not turn out to be what I expected (e.g. ANTS2)

 

 

 

 

 

2.4 Risk Analysis And Management

2.4.1 Identification

 

Risk analysis is important in any project as it allows the identification of risks. From here you can find out the likely hood of the risk occurring and how much impact it will have to the project.

 

Table 1 identifies the main risks in the project

Risk

Probability / Likelihood

Impact to project

Implementing too many features leads to project schedule over running

35%

35%

Performance requirements not met. (Software is too slow)

25%

40%

External components don’t meet up to expectations.

15%

10 – 50%

Implementing too many features produces poor quality software.

15%

60%

Project schedule over runs due to other university commitments (other assignments).

20%

35%

Poor user interface

10%

10%

 

The risk identification clearly shows that the main risks are time (/project schedule over running), performance, and external components. Therefore, I will need to do try to resolve theses risks by doing risk management.

2.4.2 Time / Project Schedule Over Running

 

Well the number one risk without a doubt is time. Will I be able to finish the project on time? Well there are many ways of reducing this risk

¤                  Start on project early. (Risk contingency)

¤                  Use external components. (Risk avoidance)

¤                  Reduce scope. (Risk avoidance)

¤                  Prioritise my work to ensure important modules are done first.

2.4.3 Performance

 

Producing a prototype of the auto completion at start of project discovered that performance was a very big risk. Surprisingly at the beginning of the project, I did not consider this being one of the main risks.

 

Luckily that I decided to make a quick dirty prototype of the auto completion feature and find out that auto completion is a real-time constraint, and the algorithm used in that prototype was useless and so needed to reassess the design. However apart from the core algorithm, I was able to keep the rest of the code, via rerfactoring . Please read in appendix B on page 68 for more information.

 

Ways of reducing the risk of performance (i.e. that the auto completion feature isn’t too slow). Is to do the following

¤                  Reduce the scope of auto completion. More simpler it is, normally quicker it will be (Risk avoidance)

¤                  Use an incremental algorithm; therefore only update the text that has changed.  (Risk avoidance). However, this will increase development time.

2.4.4 External Components

 

As this risk is low-medium risk, and has low-high impact. I will use the following strategies to deal with the risk

¤                  Do not use external components for any core modules. (Reduce Impact)

¤                  If the component does not work, find another one. (Risk contingency)


3.0 Requirement Specification

 

3.1 User Requirements Definition

 

1.       Produce a simple text editor

1.1.    Software should be able to load, save and edit text/java files

1.2.    Should have undo/redo facility

1.3.    Have copy/cut/paste option

1.4.    Should have search/replace option

 

2.       Software should help the user type in java

2.1.    Should include syntax highlighting

2.1.1. Keywords, string, single and multi line comments should be highlighted.

2.2.    User should also be able to produce javadoc of there code

2.3.    Can compile java programmes

2.3.1. Should be able to compile only the classes that have changed

2.3.2. Should be able to allow the user to view error messages from compiling

2.3.2.1.  A hyperlink should allow the user to go to the java file with that error

2.3.2.2.  If java source code is unavailable, then decompile the class to get the source code

2.4.    User can run there java programmes

2.4.1. The input/output of the running process should be displayed

2.4.2. Should be able to run multiple instances of there java programme

2.4.3. Software should automatically search through code and give a list of java files that are runnable

 

3.       Should include auto completion facility

3.1.    This occurs when user types in a variable name or static object

3.1.1. Software should parse the java code to get all possible java classes from the import statements.

3.1.1.1.  Before hand, you will need to know what classes are available from the java runtime, and from any external libraries.

3.1.2. Java code needs to be parsed in order to identify variable names and declarations

3.1.3. Incremental parsing should be used

3.2.    All the methods and fields for that specific object should be shown

3.3.    As the user types in the text, only the possible methods/fields should be displayed

3.3.1. For example if user typed ‘System.o’, then it will only show methods/fields for the System object which has the letter ‘o’ at the beginning (i.e. the ‘out’ field).

3.3.1.1.  The javadoc for the method/field should be displayed

3.3.2. Alternatively, if the user types ‘new’, then all the possible classes/objects should be shown.

 

4.       Java work should be grouped into a project

4.1.    This contains a directory called ‘src’ where all the java source code remains

4.2.    Should contain a directory called ‘build’ where java classes are stored

4.3.    Should contain a directory called ‘docs’ where the javadoc is stored

4.4.    Also a directory called ‘lib’ where any external libraries can be stored

 

5.       Project should be stored on a server with remote access

5.1.    Files should be cached locally for fast access

5.2.    Should be optimised for low bandwidth

5.2.1. Only files that have changed should be transferred

5.2.2. All transferred files should be compressed before being sent

5.3.    Multiple clients should be able to simultaneously contact server

5.4.    No two clients can edit the same java file

5.5.    Clients should only send files to server, when a client quits the IDE or when java code has been successfully compiled.

5.6.    Software should be able to go through university firewall

5.6.1. Must implement an HTTP protocol

5.6.2. Also implement SSH protocol

5.7.    Full duplex communication should occur between server/client

5.8.    Client should be able to chat to other clients on-line

5.8.1. Instant messaging should occur something similar to MSN messenger or yahoo chat.

5.9.    Server should be able to run via command-line with no GUI

5.9.1. This allows users to run the server via telnet

5.10.Should implements an asynchronous protocol

 

6.       Should integrate useful external components to help the user

6.1.    Integrate Log4J with Lumbermill

6.1.1. This allows the ability of logging for the users

6.2.    Integrate PMD to allow the user to review their java code.

 

7.       Should help user to import java code/classes/jar

 

8.       Should help user create a new java class

8.1.    Should include if user wants a main method

8.2.    Should include option of logging

8.3.    Should include option for default constructor

3.2 Non-functional Requirements

 

9.       Javadoc should be very easily accessible and integrated with the software

9.1.    A mini web browser should be integrated to allow easy navigation of javadoc

 

10.   Software should store any previous information which might be useful in the future

10.1.Should have recent project list

10.2.Should store look and feel of software

10.3.Should store the settings for server/client

 

11.   User should have the ability to change the look and feel of software

 

12.   User should be able to view java doc while seeing their java code.

3.3 Efficiency

 

13.   User should be able to open/save/compile/run there work with no more than 3 ‘clicks’

 

14.   Server should be able to run comfortably with up to 10 clients.

 

15.   User should wait no more then 1 seconds for the auto completion to work (on either the NT/Linux computers in University)

 

3.4 Portability

 

16.   Software should be able to run on any machine with java runtime 1.4 or above

16.1.Software should 100% pure java

16.2.Should not contain any native coding.

 

17.   Can run software from a website

17.1.Software should be able to be executed via an applet

17.2.Software should be able to be executed using JNLP

 

3.5 Prerequisite

3.5.1 Software

·         Required java 1.4 runtime

3.5.1 Hardware

·         Internet access -           Required for loading the java doc via the sun website

·         Network  -                    Required for server/client

 

3.6 Attributes

 

Attributes

User Requirement No.

Comment

Status

All

Approved

Priority

1, 2, 3 – Mandatory

5 – Useful

6 – Optional

Fundamental to project

Effort

All

 

Risk

3 – Medium-High

Task of incremental parsing is very difficult.

Stability

All

 

Target Release

All

Development must be completed by March 2003.

Assigned to

All

Mohammed Imran

 

 

 

 

 


4.0 Design

4.1 Overview

 

As I have already discussed in section 2.1 Choosing a software development methodology/model I will be using incremental software development model.

 

So the purpose of this section is to produce a top-level design, which can be used to break software down into increments/modules. So in this section I will describe the overview design and architecture of my project. It will initially start with a naive design, which I will slowly expand, and explain until a complete design has been made.

 

4.2 Simple Design

 

Figure 4 shows the external interactions of my project

The best way to produce a piece of software is to break it down into modules, which have high cohesion and low coupling.

 

My aim in this section is to break the project down into modules, which have the bare minimum interactions with each other.

 

This may result in having many modules, but it works very well with incremental development.

 

Note that all diagrams in this chapter that have the colour blue represent a module.

 

 

Figure 4 shows the interactions between my IDE and external components. Note that I have separated the firewall into two parts, this was done to avoid confusion and emphasis the fact that the client and server communicate through the firewall, but the IDE cannot communicate at all to the server. In later diagrams there will be only one firewall, which is the case in real life.

 

 

Now in order to be able to finish this project on time, I will need to use external software to finish some of my modules, however I will implicitly state which software was used and why I decided to do this. This will mainly be done in the later chapters.

 

 

As you can see from Figure 4 I have not given any details about the internal design/structure of the IDE, however you do know that it connects to the Internet in order to get the java documentation. It can communicate to the client and has direct access to project data.

 

 

The project data is merely a directory, which contains all the vital information for the project.

2

File

config.lexx

Contains settings for the project, such as colour preferences, or which classes the user can run.

1

Dir

Src

This directory in which the source code is stored.

1

Dir

Build

All classes are compiled and stored here.

1

Dir

Docs

The javadoc produced from the source code is stored here.

1

Dir

Lib

Any external libraries or classes that the user wants to add to there project will be placed here.

 

You will notice in Figure 4 that there is client/server relationship that can be placed in a module called client/server. The general purpose this module is to allow remote access to the project data. The idea of this is that the project data is stored on a server. Let us say at university, and then the client interacts with server to update the local project data.

 

Note to achieve the bare minimum interaction I ensured that the IDE could not contact the server. I’ve also ensured that the client automatically updates the project data without the IDE knowing. The only interaction the IDE has with the client is when it wants to load a java file up, it must first ask the client. Or when the IDE has finished with a java file it must tell the client that the file is free. The IDE will be automatically given the client; it doesn’t need to create it.

 

In order to achieve instant messaging between clients and have low coupling. The client will merely give the IDE a Panel to display. The client will handle the entire instant messaging stuff itself.

 

So fundamentally, I have broken the project down into two sections a client/server and IDE. However the IDE can still be broken down further.

 

Figure 5 shows the simplified and incomplete architecture for the IDE

In this chapter I will sometimes refer to the requirement specification to give an indication what the purpose of the module is. However it does not mean that the module in question can fulfil those aims. For example the details in the specification 5.x do correlate well to the purpose of the client/server module, but in order to meet 5.x requirements numerous modules (including client/server) will need to work together to fulfil the requirement.

 

 

I will now start to look at the internal structure of the IDE, which begins in Figure 5. I have tried to break this down into distinct modules. (The files for java runtime libraries can be found in the java system properties in ‘sun.boot.class.path’.) Figure 5 breaks down the ‘java IDE’ found in Figure 4, into the following modules.

 

 

GUI module

 

This wraps all modules together inside a user interface. It also includes GUI into create/import new java files. (Please refer to 7-11 in the specification, to give the idea what else the GUI may need).

 

As nearly all modules interact with this one, and this being a wrapper module means that the GUI module will be continuously modified throughout the project. In addition, the GUI will normally always done after a module has been completed.

 

 

Java editor module

 

This module actually edits the source code and provides undo, redo, cut, copy, paste, and syntax highlighting. Please refer to 1-2.1 (page 10) in the specification.

 

The java editor interacts with both the GUI and the auto completion module. All the icons/buttons such as undo, redo, cut, copy, paste will be handled by the GUI module. Therefore the java editor will need methods accessible to the GUI (for example cut(), copy() and so on).

 

The java editor also interacts with the auto completion module. The idea behind this is that the java editor will create DocumentListeners, which represent the auto completion module. (A DocumentListener sends a DocumentEvent every time the user edits the text.)

 

 

Auto Completion module

 

The auto completion module implement DocumentListener, and perform object auto completion.

 

 

Compiler/runner interface module

 

This module is a user interface for the external compiler Apache AntS2, or any running JVM processes.

Apache Ant can compile and produce javadoc by creating a ‘build.xml’ file that Ant uses. The user interface will need to handle the output produced by Ant or any other running process. The output (the text displayed such as an error message) will be placed into a textpane, which will contain hyperlinks to any references to the source code. To give an idea of what this module does and why it is necessary please refer to 2.2-2.4 (page 10) in the requirement specification.

 

The way to reduce coupling between the compiler/runner module and the GUI, is to let the compiler/runner module handle everything. Therefore, the GUI only needs to add panel to itself, and tell the compiler/runner module what it wants to do. Such as compile, produce javadoc, or run a java file.

 

 

External plug-ins interface module

 

This module is a user interface to the external components such as PMD, JRefractory, JODE, and L2FPROD. More details in later chapters, however requirements 6.x will give you a good idea what the module is suppose to do.

 

However, this module only interacts with the GUI, and in all those cases the GUI merely creates/calls the plug-ins. Therefore in-depth analysis isn’t required.

 

 

 

Javadoc interface module

 

This is a user interface to a web browser called Calpha HTML. This views javadoc from either the suns website or the javadoc found in the ‘docs’ directory. Once again this module has some correlation to the requirements in 9.x.

 

The javadoc module merely requires the web address, and all it does it returns a panel, which the GUI adds to itself. As a result coupling is put to the bare minimum.

 

 

Note that nearly all the modules will need to know the directory name of the project data, as they do their own modifications.

 

Instead of having the GUI to pass this information to every module, I will instead use a static factory class, (which in my project is called Config) to store this information, and all the modules can call the factory class.

 

I’ve also applied the same idea to the javadoc module. Instead of asking the GUI every time to tell the javadoc module to load up a web page. There is a static factory class (Config) which loads up the page. Therefore, the auto completion module can indirectly tell the javadoc module to load a page, but cannot access the panel/javadoc module.

 

 

 

 

4.3 Auto completion Design

 

This module will implement DocumentListener, so the java editor module can use it.

 

Note that the auto completion module is still very big and can still be broken down further. The auto completion requires many individual steps, which is briefly discussed below:

 

1.       Read in all classes from the java runtime libraries and any external libraries.

2.       Put all the classes into a class table

3.       Have an incremental parser that takes in java code and produces an abstract syntax tree (AST).

4.       Have an ‘analyser’ which identifies all possible classes via import statements in AST and by using the class table.

5.       Analyser should be able to get all possible variables names or static objects by looking at the variable declarations made in the AST.

6.       Analyser at a given position should use the information in step 5 to work out if a class/object exists at that position.

7.       Get the object (if any) at a given position from the analyser, and then show all the methods and fields for that object.

8.       Allow the user to select the method/fields.

 

 

 

 

Figure 6 translates these steps into smaller more manageable (and cohesive) modules.

 

 

 

(Note that I tried to keep the explanation of auto-completion at simple as possible, for the design phase. A much more in-depth explanation can be found in later chapters).

 

 

Figure 6 shows the architecture of the auto completion facility

 

As you can see from Figure 6, it now shows these new modules:-

  • An API Reader module – which performs step 1 and creates a data structure containing all possible classes available. Please refer to 3.1.1.1 in requirements for an alternative explanation.
  • Class Table – This performs step 2. This data structure needs to be able to stores a large amount of data with fast access time.
  • Parser – Performs step 3. (Please refer to 3.1.3 in requirement for slight insight of this module).
  • Analyser – Performs steps 4, 5, and 6. (For more information look at 3.1.2 in requirements).
  • Auto Complete module – This does steps 7, and 8. (Please refer to 3.2 and 3.3 in requirements).

 

If you would like to have a much more complete overview design, please refer to appendix A on page 67. However, as project is quite large, this can look somewhat overwhelming, this is why I recommend that you refer to Figure 5 and Figure 4.

 

Now the project has been sufficiently been broken down into manageable tasks. I can now start to look at each module separately and use the incremental software model.

 


5.0 GUI Module

 

5.1 Overview

 

In the section 1.2 Other Students Thoughts, one of the main criticisms of XEMacs was the user interface, this is why I must ensure that the GUI is user friendly as possible.

 

The point of this module is be the GUI for the most of the other modules, and to perform simple tasks such as create a new class file or to be able to import files to a java file. Show a list of all the java files, and to setup all the classes for the other modules.

 

5.2 Feasibility

 

First thing first, I need to have an idea of that the GUI should look like.

 

Figure 7 shows the main GUI for my IDE

 

Well firstly, in order for the user to be able to quickly switch from one java file to another I have decided to place all the java files in a JTabbedPane, this can be seen above in Figure 7.

 

 

Now for seeing the panel for javadoc, or when compiling the users work. I have decided to place all of these panels on the bottom of the screen in another JTabbedPane. The reason for this is that the user will be able to look at their java code while looking at the javadoc or an error message in the compiler.

 

 

In addition, the user should be able to load up their java files in less then three clicks. In order to achieve this, I have decided to put a JTree on the top left hand corner of the screen, which shows all the files in the ‘src’ directory. Therefore, the user can easily click any file he/she would like to load up.

 

 

In any good GUI, there should be a toolbar with the most used functions placed on it (and a tool tip to explain what they do). Most used functions should also have shortcut keys for advance users and there should be menu bars for all the other functions.

 

 

In addition, the look and feel of the IDE should be similar to the operating systems look and feel. Unfortunately, most users can tell when a programme is written in java even when you set the look and feel to the operating systems. (The user knows that something does not look right. Especially if the operating system is windows XP or Mac OS X). This is why I have decided to implement the option to change the look and feel of the IDE to the users own personal preferences. This also includes the ability to add their own skins to the IDE. This can be achieved by using an external piece of software called L2FProd S10, which takes in a theme pack (which are freely available from there website), and all I need to do is add a few lines of code to my GUI to load the file from the theme pack. This is the quickest way to achieve this.

 

 

So far, I have discussed the main screen in the GUI, but I haven’t discuss the other dialogs that are required such as create/import a file, or to load up the project first. To create a java file the user should be able to doing the following options

 

·         Choose which package the java file should be stored

·         Choose the name of class

·         Have the option to create a main method

·         Option to enable logging

·         Option to create a default constructor

 

 

Once this is done the java code should be generated, this is a quite a simple thing to do. Please refer to Figure 8 to see what the dialog should look like.

Figure 8 shows a screen shot of the create new java file dialog

 

 

As I have already said the user should be able to import files. In order to achieve this there must be some sort of file chooser, which allows the user to select the files he/she wishes to have. Please look at Figure 9 to give an idea what this should look like. Once I have a list of files that the user wants to add, I will then need to work out where these imported files should go, for example if it’s a java file then it should go to the ‘src’ directory however if it’s a class/jar file then it should go to the ‘lib’ directory. In addition, the user should have the option to select a directory, and all of its content should be imported,

 

 

 

 

 

 

Figure 9 shows a screen shot of the import files dialog

 

5.3 Implementation

 

Creating the toolbar and the menu items are trivial tasks, but they do take up a lot of my time. This is why I decided to use JBuilder to help me to create these menu items for me, in order for me to keep on schedule. However all other aspects of the GUI, was done completely by my self without using any build tools.

 

As the GUI is a wrapper for other modules such as compiler/runner, javadoc and editor modules. There will be some cases where I need to shutdown/close one of these modules down. For example if the user want to close a java file or when the user wants to exits the IDE. It would be disastrous if the GUI just closed down the java file without asking the user first if he/she would like to save the file first.

 

This is why I have ensured that nearly all the other modules must implement an interface called Closeable. All this contains is a single method boolean canClose(), in which the GUI calls the method to finds out from the other module if it can shut it down.

Figure 10 shows the main classes involved in the GUI module (excluding all the classes from the other modules). You’ll notice that the main GUI is in a class called EditorFrame, and that implements ShowJava. ShowJava is an interface, which is required in the compiler/runner class, it allows me to load a java file up and go to a specific line.

 

In addition, the user should be able to choose which java files to run. Now the user-friendliest way to achieve would be to show a list of all the files that a runnable (i.e. contain main method), and the user merely selects the class he/she wants to run. This is done in the RunPanel class, and the way it finds out if a class contains a main method is by looking in the ‘build’ directory and going through every class file finding if it contains a main method.

 

You’ll probably think it would be simpler to read the source code and find out if it contains a main method instead of reading the class file. However reading the source code does not take into account inheritance, a java file could extend a class with contains a main method. Only reading the class file would find this out.

 

 

 

 

 

 

 

Figure 10 shows class diagram of the GUI module

 

One thing I have missed out is show the user first chooses or creates a project. This is actually done before EditorFrame/IDE is even loaded up. I decided to create a class called startup?????????, which gives the option to load/create a project, but also allows the user to connect to a server, or to make the current machine a server. Please look in Figure 11 to see the start-up dialog.

 

Figure 11 shows a screenshot of the start-up screen

 

 

The startup????????? class will also have a recently used list, to remember previously loaded projects (which is good HCI). Also the startup????????? class actually creates the Packages class (API module required in auto completion), and so the EditorFrame class is already given all this information.

 

 

5.3 Testing

 

Testing for this module was quite difficult, as the best way to do testing is to create automated test cases (please refer to 12.1 Automated Testing - JUnit S1on page 53). However, as these tests involve a visual checking it is not possible to automate it, and so I have to manually need to test the GUI every time I make any major change to the GUI.


 

6.0 Java Editor Module

6.1 Overview

 

The point of this module is to produce a simple text editor with syntax highlighting support. Therefore, it should have the following options

a)      Undo and redo option

b)      Have the ability to cut, copy and paste code

c)      Have the ability to find and replace text in editor

 

6.2 Feasibility

 

Well there are many ways of me being able to create a text editor. For example I could attempt to create the editor completely from scratch i.e. do not use any javas swing/awt components.

 

On the other hand, I could extend JTextArea to implement my addition options. However, java does have another component specifically made for people who want to write an editor, this is JTextPane/JEditorPane. These components already have in-built to include the cut, copy, and paste options, and they can easily be modified to support undo and redo. Therefore, to produce my editor I will be using JTextPane.

 

6.3 Implementation

 

I have decided to have a class called EditorPanel which extends JPanel and inside this class contains JTextPane. In order to achieve undo/redo I used a class called UndoManager (javax.swing.undo.UndoManager) that records changes in the JTextPane, and all I need to do is to create listeners for UndoManager to perform undo/redo.

 

Figure 12 shows the class diagram for this module. You notice that the listeners that I was referring to are called UndoAction and RedoAction. You will also have noticed that there are two abstract classes AutoCompleteListener and ParserListener, now these two listeners are required to enable auto completion for the editor. All the editor needs to do is add AutoCompleteListener and ParserListener to the JTextPane. The auto completion module handles everything else.

 

The other features required editor module is the ability to show the line number, which is useful as messages tell the user the line number as well.

 

The only thing that is left is syntax highlighting. Now originally, when I was creating my proto-type during the beginning of the project I wrote the syntax highlighting method from scratch please look at appendix B on page 68. Needlessly to say I broken down the code into tokens, and I decided to perform syntax highlighting by using the StyledDocument class.

 

The StyledDocument class changes the JTextPane from a simple text editor to a much more powerful editor, in which you can include images, components, and coloured text inside the document. Even though my current project were not intending to have components/images inside my editor, I was thinking about the future and if the project was ever extended then these option would be useful, and so I decided that I would use StyledDocument class to do my syntax highlighting. All I need to do is tell the StyledDocument the region of text I want to syntax highlight, and the colour (AttributeSet) that I would like to change the text to. I only need to specify this once and it will continually remember the colour settings for that text

 

 

This is much more simplier than my alternative option which was to use a PlainDocument which doesn’t give any support for coloured text, and so I would have to manually have to change the paint method in the PlainDocument class. This is achieved by extending the View class. However this is a very tedious task as I have to continuously redraw the coloured text every time the paint method was called, and as my project schedule was tight I decided to go for my first option i.e. use a StyledDocument class.

 

This worked perfectly fine in my proto-type however as you can read in appendix B, my proto-type was too slow, however I made the big mistake in assuming that the main bottle neck to my proto-type was my parsing algorithm. This was unfortunately not the case, StyledDocument is very powerful compared to PlainDocument, however this came with a cost, which was performance, and found this out near the end of the project. As my project schedule was already very tight, I did not have enough time to re-write my syntax highlighter to use PlainDocument instead.

 

Figure 12 shows the class diagram for the editor module

 

So in order to have syntax highlighting and to ensure that it does not slow my IDE down I decided to use an external piece of software called Syntax Aware Text ComponentsS5 (SATC). This basically performs syntax highlighting by using PlainDocument and extending the View class. The only advantage that this software had over my algorithm (apart from performance) is that it works on any parser that was written in ANTLR. (For more information about ANTLR please look at 7.5.3 Parser Module on page 36), whereas mine was specific to java. Nevertheless, saying that SATC does not work well if the text is incompletion, for example, it will not recognise if something is a string if it does not end with “, whereas my algorithm will.

 

This is one reason why I have decided to keep my syntax highlighting algorithm in my code, because if the project was ever extend I would simply update my syntax highlighter to PlainDocument, but until then I will continue using SATC. (If you would like to enable my syntax highlighter please add this line before running my IDE ‘–Doldsyntax=true’.)

 

Note that SATC hard-coded where the ANTLR parsers were, therefore I had to change the source of SATC, to ensure it was more dynamic.

 

As the editor module was eating-up more of my project schedule I decided to save my self some time and use external piece software called Find & Replace Text Package S4 to perform find/replace in the editor. All it does it display a dialogue box, in it updates JTextPane.

6.3 Testing

 

Testing for this module was similar to the GUI, in which tests involved visual checking which as a result doesn’t allow automate tests. Apart from the syntax highlighting algorithm which was in my first proto-type.

 


7.0 Auto Completion Module

 

7.1 Overview

 

The point of this module is to perform auto completion on the java code. This will occur in two cases

         i.            When a user is about to call a method/field in an object. The auto completion should start when the user presses the ‘.’ character next to an object.

       ii.            The second occasion occurs when the user types in the keyword new and all the possible classes/objects are displayed in a popup.

 

These are probably the two most useful auto completion features I have seen. All others features are somewhat trivial and do not help the programmer that much.

 

Now even though the point of this module is to perform auto completion on java code, I will attempt to design my module is such away that it can easily be extended to any other object orientated language. Therefore, you could also do auto completion for C, C++, small talk, and so on.

 

 

7.2 Feasibility

 

The first step is to find out what are the inherited similarities that java has to object-orientated (OO) languages.

 

Fundamentally OO encapsulates variables and methods/procedures into an object. Now these methods or fields can return other objects. All OO languages have a tree like structure to the source code. In the case of java, it starts with class, which can contains methods and/or variable declarations, and within these methods contains statements, and/or inner classes.

 

Therefore, the first two steps of auto completion for OO will be

         i.            Getting all possible objects

       ii.            Converting linear text (source code) into a tree like structure.

 

The next step will be to workout if an object exists at a certain position. If it does then we can show all the fields/methods for the object. Now this is a very difficult thing to do.

 

Figure 13 showing an example of some java code

package testing;

import java.io.*;

import java.awt.*;

import java.net.*;

 

public class Example extends Panel

{

  int num = 50;

  public String getURLName(String name)

  {

    String filename = name + num;

    File f = new File(filename);

    return  f.toURL() .toString();

  }                  á

}                    position A

 

For example, lets say we want to do auto completion at position A of Figure 13. So the user types ‘f.toURL().’, and what should happen is that all the method/fields for the object java.net.URL should be displayed.

 

I hope that by now you would have noticed that this problem is a similar problem that are faced by compilers, and since compilers has been a researched topic ever since the 1960’s; there is good documentation and support. As a result, I will try to see how a compiler would solve this problem.

 

Figure 14 shows the steps involved in compiling source code

 

The compiler is broken down in many stages, which can be seen in

Figure 14. All that we are interested are the first 2-3 steps, i.e. lexing, parsing and analysis. These fundamentally break the code down into an abstract syntax tree (AST). The way the compiler would workout the object at position A in Figure 13 would to be to walk though the AST storing any object/variable declarations in an environment/symbol table until it gets to A.

 

For example at position A the compiler would have the following environment/symbol table.

     num                 :: int

     filename      :: java.lang.String

     name               :: java.lang.String

     f                      :: File    (which is actually an abbreviation of java.io.File, which needs to be                                  worked out from the import statements, however this problem is specific to java)

 

Once it gets to line A ‘return f.toURL().toString()’. The AST would have already broken down this line into subunits. Some like [return [ f. [ toUrl(). [ toString() ] ] ] ]. The compiler would then go through each subunit. It first ignores return. Then it sees f, which corresponds to java.io.File from the table. It then gets all the methods/fields for object java.io.File and adds it to the table. It then analyses the next subunit, which is toURL(), which corresponds to java.net.URL in the table. So at position A the object is of type java.net.URL.

 

This does sound like a complicated solution, however in reality this the most efficient way of doing it. Therefore, in my case I will use the same principles that compilers use to solve this problem, but I’ll need to just make some adjustments for it to work in my case. Figure 15 shows these changes.

 

Figure 15 shows the individual steps to perform auto completion for method/field names

a)      Have an environment/symbol table of all possible objects.

b)      Have an incremental lexer/parser to produce an AST. – The reason for it being incremental is that if I had some code, which was 100Kb large, then every time the user presses a key it would need to parse 100Kb of code. This would most defiantly reduce performance; instead, it should only parse the area of code that has changed and then just update the old AST.

c)      Have an ‘analyser’ which we will call ASTAnalyser, which takes in both the AST and environment/symbol table, and returns the new environment/symbol table at a specific position of the code (lets call that position X) via tree walking.

d)      Then have another ‘analyser’, lets call it AutoAnalyser, which would walk backwards from position X, until it finishes the statement.

 

In the case of Figure 13, AutoAnalyser would go backwards from position A, getting

.ŕ).ŕ().ŕL().ŕ …….. ŕf.toURL()ŕ  f.toURL()’, end of statement.

e)      AutoAnalyser would break the statement down into sub units. Then just like the compiler, it would go through each subunit identifying the object, and adding the methods/fields to a new environment/symbol table.

f)       AutoAnalyser keeps on doing this until if finishes all the sub units. If an object is found then display all methods/filed of the objects.

 

The steps in Figure 15 are more complicated then the compiler method. However if you compare Figure 16, and Figure 17, you’ll notice that in Figure 17 ‘f.toURL().’ was inserted. Problem is that this line is incomplete and so the incremental parser cannot parse it. Therefore, incremental parser will only have its old tree, which is the AST from Figure 16.

 

Figure 16 showing an example of a complete java code

public class Example2

{

  public File getFile(String n)

  {

    File f = new File(n);

  }

}

Figure 17 some additional text being added to Figure 16

public class Example2

{

  public File getFile(String n)

  {

    File f = new File(n);

    f.toURL().

  }

}

 

This is why I need AutoAnalyser, to analyse the line that the incremental parser could not parse, and break it down into sub units.

 

Now this would be the best way to do auto completion in an OO (or even other non-OO) languages. Note that producing a parser can be very complicated, but once again as compiler is an active research topic there are also many parser generator tools, specially made to help in compiler construction. I could use these tools to produce my AST. All I need to do if write the grammar for the specific OO language, and the parser generator takes this grammar can converts it to parser.

 

In the case of this project I’ve decided to go for an LL(k) parser generator called ANTLR, which is open-source, and has great support. My backup if ANTLR does not turn out to what I expected is JavaCC, which was initially produced by Sun. However, JavaCC does not have as much support compared to ANLTR.

5.3 Design

 

Now I have decided how to do auto completion for OO. I now need to design it. In the previous section I have been referring to an environment/symbol table, which contains text/key that corresponds to type/object. The best way to represent this environment/symbol table is to use a trie data structure.

 

 

Basically trie stores objects in a tree like structure. Figure 18 best illustrations this.

 

So in a trie data structure I would simply call the method get(String key), and it returns the corresponding object (or null if it doesn’t exist).

 

Now for the purpose of my project, I will need to be able to store multiple objects with the same key. So instead of returning an object, I will return a list of objects.

 

In 99% of cases these will be a list contain only one object.

 

My first step in auto completion is to get all possible objects. This job will be designated to a class called Packages. This is required for analysing variable declarations.

Figure 18 showing an example of a trie data structure

 

I will now take the steps in Figure 15 and translate them into classes. Figure 19 shows this translation, it also gives an indication of the flow of data, from one class to another.

 

Figure 19 showing simplified diagram of the classes involved in auto completion

 

 

As you can see in Figure 19 auto completion is triggered when an event is given.

 

This event comes from DocumentListener. This basically gives creates n DocumentEvent every time a user inserts or removes something in the document/text.

 

 

 

 

Figure 20 shows an alternative view to the diagram in Figure 19.

 

 

Therefore both classes, ParserListener and AutoCompleteListener will implement DocumentListener. The purpose the class ParserListener is to perform incremental parsing, and produce an AST for the ASTAnalyser.

 

The ASTAnalyser takes in the environment/symbol table of all possible objects (in the form of Trie) and then creates a new environment/symbol table for a given position, which the AutoCompleteListener uses.

 

Finally AutoCompleteListener takes this environment/symbol table and uses it to workout if an object exists, if so it sends the possible methods/fields to the SelectionPopup.

 

The SelectionPopup is the GUI side of auto completion, it displays all possible options and inserts the text back into the document, once user has decided which method/field they would like.

 

Figure 19 merely shows the classes that are involved in auto completion, however it doesn’t show the methods required for each class. This can been seen in Figure 21 which shows a full class diagram of auto completion.

 

In all of the class diagrams in this report are colour coded. Blue classes represent interfaces, whereas yellow ones represents an abstract class. A white/clear class shows a normal class. A green arrow represents inheritance. Blue arrow shows generalisation, and finally a purple arrow represents an instance of classes. Any text that is italic/slanted represents an interface method.

 

The first thing you notice in the class diagram is that all the classes involved are either interfaces or abstract classes. This is the case, because every OO language is different in some form its impossible to write a universal one. However, as I already said OO languages also have many similarities this is why I can write the code for some methods but not all of them.

 

I will now discuss the main methods involved in each class. For packages, I have decided to put this in an interface since the method of getting all possible objects dramatically differs from one OO language to another.

 

The Packages class contains the following methods:

 

            Trie getClasses()                       - This stores all the possible objects (including the full name) in a trie data structure. For example

           java.lang.System                        :: java.lang.System

           java.lang.String     :: java.lang.String

 

            Trie getEveryPossibleClass()     - This is similar to getClasses(), however it stores the full and short name for the object/class. For example

           java.lang.System                        :: java.lang.System

           java.lang.String     :: java.lang.String

           System                                   :: java.lang.System

           String               :: java.lang.String

 

            List getClassMap()                    - This stores all the possible classes in a list.

            void update(File f)                     - This basically updates get methods to include this object.


 

Figure 21 showing (UML) class diagram for auto completion

 

The ParserListener is an abstract class what requires the method update(DocumentEvent chng, boolean isInserted), to be filled in. The point of this method is to perform incremental parsing, this is achieved by calling the updateTree method in the class IncrementAST.

 

You’ll also notice that IncrementAST extends AST (class from ANTLR).The reason for this is that you can use ANTLR to generate your parser for you. However AST doesn’t contain the position of the text, which is required for auto completion. This is why I have to extend AST to accommodate this. You’ll notice that IncrementAST is abstract as well, which requires these two methods to be filled in.

            int[] updateTree(String code, int offset, int lengthOfChange) , and

            List walkThruTree(int pos)

 

updateTree actually does the incremental parsing, however as every language is different, so will the way of doing incremental parsing., however in order to achieve this you’ll most likely need to use the helper methods that I have written in IncrementAST

 

Now the walkThruTree method is required for the ASTAnalyser class. This method converts a tree like structure into a linear list depending on the given position.

 

AutoCompleteListener requires the method update(DoucmentEvent e, boolean wasInserted) to be filled in. This performs the job of AutoAnalyser in Figure 15, i.e. given a position it slowly goes through the statement finding out what object exists.

 

Finally if AutoCompleteListener does find an object it sends all the methods/fields in the object Trie to the SelectionPopup class.

 

SelectionPop is an interface, which handles the GUI side of the auto completion. This is where it displays the methods/fields, and updates the display when user enters more text. SelectionPop is the class that will enter text back into the document.

 

Note every time the user presses a key, the AutoCompleteListener will call the update method in SelectionPopup if the display is already showing.

 

Now I have designed auto completion for OO languages generally. I will now start to look at auto completion specifically for java.

 

7.4 Java Auto Completion

 

From 5.3 Design, all I will need to do is extend or implement the following classes

            Packages                                  to         JavaPackages, (API module)

            ParserListener                          to         JavaParserListener, (Parser module)

            ASTAnalyser                            to         JavaASTAnalyser, (AST Analyser module)

            AutoCompleteListener               to         JavaAutoListener, (Auto Completion Analyser module)

            SelectionPopup                          to         JavaPopup, (Auto Completion Analyser module)

            IncrementAST                          to         JavaAST. (Parser module)

 

The class diagram for java auto completion can be seen in Figure 22, it includes the above classes.

 

In the next section I will explain how I went about implementing the above classes/modules.

 

 


 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Figure 22 showing the class diagram of the java auto completion module


 

7.5 Implementation

 

7.5.1 API Reader Module/JavaPackages

 

Point of the API reader module is to produce a class called JavaPackages, which contains all the possible classes that are available to project/JVM.

 

First thing first, I need to find out is how do I get the methods/fields of an object, and the name of the object. Fortunately java stores all this information in the reflection package (‘java.lang.reflect’), and that every object inherits a method called getClass() which returns an object of type Class. This Class object contains all the information that is require for auto completion3. (To help to avoid confusion between the object java.lang.Class, and class in the OO sense, I will highlight java.lang.Class in blue)

 

Therefore, all that needs to be done is to store the Class object for every possible class. Only problem is that Class object can take up a lot of memory so instead I will create a class called ClassName, which can get the Class object.

 

ClassName takes in three parameters, first one is the name of the class/object, the second is the package that the class belongs to, and the final one is the physical location where the class is stored.

 

Classes can be stored on their own (.class), or in a zip/jar file. So JavaPackages will need to handle these types. Now for the purpose of this project all the possible classes can only come from three sources, which are:

·         Java runtime libraries – which can be found from the ‘sun.boot.class.path’

·         Build’ directory which contains the classes from the compiled source code

·         Lib’ directory, which includes any external libraries.

 

Now the difficult bit in this module is not recursively going through a directory/zip/jar file finding classes and creating instances of ClassName, which is added then to a Trie/list data structure, but instead implementing the ability to load a class file from a physical source.

 

Normally in java if you want the Class object all you need to do is call Class.getClass(String nameOfClass), however this only works if the given class is already stored in the java runtime classpath, which is not the case for my project.

 

Therefore, I need a way of getting the Class object from a physical file. Now originally I tried to solve this problem by using URLClassLoader, which is designed to do exactly what I require. However due to poor documentation of URLClassLoader I could not work out how to use it.

 

Therefore, I was forced to use my backup plan, which was to run a separate java runtime process in which I have specified the classpath, and I get the Class object via RMI.

 

This obviously complicates matters, and reduces performance. I was very reluctant to this but I could not find an alterative.

 

Near the end of the project, I finally worked out how to use URLClassLoader and due to the way, I designed my JavaPackages, all that I needed to change was one method in ClassName. So fortunately, no major rework was required, but there was a increase in performance.

 

7.5.1.2 Performance

 

As the classes ClassName, and Trie are both Serializable and Cloneable. I have implemented the ability to store the data structures found in JavaPackages into a file, and so every time the user loads up the IDE or runs a test case he/she does not need to wait for JavaPackages to read the classes/jar files again, it merely loads the file back into memory.

 

However, the classes/jar files can change in the project directory/data. Therefore, I will only store the classes that are from the java runtime libraries, as they will always remain constraint. As a result I have saved the amount of time it takes to initially load up my IDE.

 

7.5.1.3 Testing

 

You would have notice that in the auto completion section there is a tight coupling between each module.

 

However, in order for me to write automated tests I need an easy way of creating these dependent classes without the hassle of editing the settings. This why in many of my modules I have added a static method called getDefault(), which returns an object with the most common settings.

 

As JavaPackages/API Reader is the first step in the auto completion, I had implemented the getDefault() method which returns all the possible classes from the java runtime. It ignores any classes/jar files from the project directory/data. The getDefault() should only be used in testing.

 

Also I have added log information to JavaPackages and have created a class called JavaPackagesTest which runs automated test cases to check if the JavaPackges class actually works, (prefer refer to 12.1 Automated Testing - JUnit S1 on page 53).

 

7.5.2 Trie Module/Data Structure

 

As you have already seen in Figure 15 and Figure 22, the class Trie is used throughout my auto completion section, and for that reason I have decided to place this trie data structure into its own personal module.

 

Now my Trie class is not your conventual trie data structure. Firstly it stores a list of objects instead of just one object, but most importantly you would have noticed in Figure 18 and in Figure 24 that my Trie class doesn’t have any add methods.

 

The reason for this is that throughout my project, I am normally given a Trie object and I am required to add a few more objects to it. For example the environment in auto completion section, every time I go through an AST I will add variable names to an existing environment. However, when I leave a method/block of code I need to go back to the original environment.

 

Normally in java, what I would have to do is it to make a copy of the environment first before I modified it. This is somewhat of a waste of memory and performance. As am only adding to environment.

 

The ideal solution would be to have a data structure, which could still refer to the content of the old environment. This is where I decided to create a class called MultiTrie. The full purpose of MultiTrie is to contain many small Trie’s and emulate it as if it was one single Trie.

 

This same principle was applied to the Packages class, as there are two methods getClasses() and getEveryPossibleClass(). Now the getEveryPossibleClass() will contain the same exact data found in getClasses(), however getEveryPossibleClass() also includes the short-hand named for every class/object.

 

So what I can do is have 2 Tries, one which is from getClasses(), and another one which only contains the short-hand name of the objects. Therefore when getEveryPossibleClass() is called I create an instance of  MultiTrie which contains these 2 Tries. Doing this saves a lot of memory.

 

So far, I have described how you can add to an existing Trie, but I have not told you how to create a Trie from scratch. Well to achieve this you need to create another class called RootNode (which implements Trie). This class does contain the add method.

 

So in summary, if you want to create a Trie from scratch you simply create an instance of RootNode. If you want to add something to an existing Trie, then you need to first create a RootNode, which contains the objects you wish to add, and finally put both the Trie and RootNode in a MultiTrie.

 

This is somewhat long winded, but it does save a lot of memory up, and also saves some time.

Figure 24 showing class diagram of the Trie module

 

 

 


7.5.3 Testing

 

Once again, there is a class called TrieTest, which contains automated test cases for the Trie module. I have also included TrieDebugger class that contains static methods, which visually displays the content of the Trie into a JTree, which is useful for logging and debugging. In all the modules in chapater automated testing was heavily used. Please read about the Simulator that I created which helped testing this module a lot please refer to 12.4 Simulator.

 

7.5.4 Parser Module

 

The point of this module is to produce an AST from a DocumentEvent as quickly as possible. A DocumentEvent contains the complete text and the position of where the text it was modified.

 

The first thing what is required is to produce an AST.  Fortunately, I was able to use a parser generator tool called ANTLRS15 to create a parser for me, and what makes ANTLR even better is that it already has a basic template for the java grammar.

 

So all I need to do is to first find out how to use ANLTR and to modify the existing java grammar to fit my needs. The main changes that were made inside the java grammar were

·         Ensure ANLTR uses IncrementAST (which I created) instead of CommonAST to build the AST

·         Ensured that ANLTR included any interfaces, class, or super statements.

·         Enabled error handling

·         Ensured that ANLTR included braces in the AST (this is required in incremental parsing)

 

The next thing that I needed to do is to create IncrementAST class, the fundamental different between an AST and IncrementAST, is that it stores the position in the text where the token can from, which is normally not required in compiler design.

 

This is important as I need go through the AST to a specific text position, this can only occur if the AST contains these positions.

 

One major disadvantage of ANTLR is that it records text position by line number and column (e.g. line 7 column 5), where as DocumentEvent records text position as a single number (e.g. position 20). So originally, in the project I tried to convert from each format when required. However, this was dramatically making my code much more complicated and was slowing things down.

 

Unfortunately ANLTR does not give you the option to change from line number, and column to just a single position. So as ANTLR was open-source I looked inside the code and made physical changes to it so it records the text position as a single number. This was not my ideal solution, however it was the best solution out of the bunch of options that I had.

 

Once I implemented these changes, I was able to take a DocumentEvent and convert it to an AST. However I need a fast way of getting the AST when a DocumentEvent is called, this is where incremental parsing comes in.

 

ANTLR does not give any support for incremental parsing, since compilers normally do not require this. Therefore, I have to implement incremental parsing myself. This is where the major work for this module came in.

 

7.5.4.1 Incremental Parsing

 

The idea behind incremental parsing is that it takes an old AST, and looks at the changed code, and only updates the old AST with the changes.

 

 

Figure 26 shows some java code and the corresponding AST

public class Example3

{

  public void sayHi(String name)

  {

    String text = “Hello “ + name;

    System.out.println(text);

  }

}

 

Figure 27 shows the java code in Figure 26 being updated

public class Example3

{

  public void sayHi(String name)

  {

    String goodday = “Hello “ + name;

    System.out.println(text);

  }

}

 

Now in order to achieve incremental parsing the following steps should happen.

         i.            It first takes position where the changed text i.e. position 80

       ii.            It finds out that the position where the text was changed was editing an existing node.

      iii.            Then from looking at the old parse tree in Figure 27, and gets the path to that position i.e.

     iv.            Then is starts at the end of the path and works backwards until it finds a node which can contain many statements such as {, for/while/do while loops, try, catch blocks, and class/method/constructor definition. In this case it finds ‘{‘ at position 63.

       v.            It goes throw all the children nodes in the ‘{‘ trying to look for the node just before where the text was changed, and just after. In this case it doesn’t find node before but it does find a node just after which is ‘;’ at position 105.

     vi.            What normally happens is that the end-position the node just before the text was changed and then gets the starting position of the node just after the text was changed. Now since there wasn’t a node before the text was changed, it uses the parent node as reference instead i.e. ‘{‘ at position 63. Therefore it the following position 63,  and 105. It then gets the text between those positions from the source code.

    vii.            This gives ‘        String text = "Hello " + name’, it then parses this text into an AST

  viii.            I then get the position of the changed node and replace it with the new AST

     ix.            Finally I increment all positions of the nodes after the changed node.

 

That’s it increment parsing is done. These steps seem somewhat simple but in reality are quite difficult unfortunately there are still some bugs in my incremental parsing this is why it isn’t used in my final version. However even without incremental parsing my auto completion is still very fast.

7.5.5 AST Analyser Module

The point of this module is to walk through an AST tree looking for variable declarations, parameters for methods/constructors and for any methods/variables that are inherited.

 

The first thing that is required it to first work out what import statements are in the AST (as users can use abrivations for classes/objects for example JPanel instead of javax.swing.JPanel), which is easy to do as they are always the first few children in the root node. Once I’ve done that I need to work out package from the AST this is required for inheritance reasons. Once all of that is done then I can look for variables.

 

The way this is achieved is that JavaAST contains a method called walkthrough which returns a list of nodes that will be visited in order to get to a specific position. Methods/loops that are not related can easily be ignored. In each node I merely look for its type if its type if variable declaration, or parameters then I analyse the node further to find out its type and to work if there exists a class/object if so then find out the name of the variable name.

 

Once that is done then I look for variable/methods that are inherited, this can only be achieved if I know what package the AST is from and if it extends any classes. Therefore this can only be achieved if the classes that are referred are already compiled. That way if I know the class, then I can work out which public or protected variables/methods are inherited. Unfortunantly working out protected variables/methods have to be done manually as java will only show inherited variables/methods that are public. All that this requires it to recursively check each super class until it gets to object.

 

Finally I put all the methods names and variables into a Trie data structure for the AutoCompletionAnalyser to use.

7.5.6 Auto Completion Analyser Module

 

This module takes a position and works backwards until it gets a statement. Once it has done that it then breaks the statement down by ‘.’ character. For example ‘listObj.get(i  )  .toString()’ is broken down into

            listObj, get(i), and toString(), (note that all white spaces and comments are removed).

 

Then I get the Trie of all the variables names from ASTAnalyser. Find if an object exists at the first state. In this case there is an object called listObj which is of type ArrayList. It then runs a recursive method which takes each segment and tries to match it with any methods/variables from an object.

 

In this case the object is ArrayList and the segment is get(i), now since the segment has brackets it knows it’s a method, therefore it only looks for methods in ArrayList, it also count the number of parameters in the segment by number of commas. In this case there is only one, therefore it looks for all the methods in ArrayList which have the name get and takes in only one parameter. The reason why I don’t check the type of the parameter is due to that it will slow down processing, and that this is only fail if there is a method with the same name, and the same number of parameters which both return different objects, which in practice is very rare.

Once I know which method is called I then get the return type of the method which in this case it Object. Then I repeat the whole procedure again but this time the object is Object and the segment is toString().

 

Once all the segments have been analysed and a object is found. I then call the class JavaPopup, which takes in a class and displays all the methods/fields for it.

 

As the user type in the characters JavaPopup will reduce the possible methods/fields so it is easier for the user to find. If the user types in any keywords like comma, open parencases, enter ,space or semi-colon. JavaPopup will automatically fill in the method/field. If it’s a method and the user type in ‘(‘ then it puts the method name with ‘(‘, however if the user types in enter, space or semi-colon is then fills everything in, which includes open and closing brackets for methods and a ‘;’ character at the end.

 

However if the user types in a dot, then the return type of the method/field is used to display a new popup will all the methods/filed of that object. There is now point in re-analysing the code if I already know what the object it. (That can dramatically increase performance).

 

That’s it auto completion is done. The screen shot on the left shows auto completion in action.

 

 

7.5.7 Performance

 

The trick of getting performance for this module is to only parse the code when you really need to. For example if you user is editing a region of code which couldn’t be parsed in the past. For example

‘JPanel p = new JP’, then I can easily know that unless the user types in either of the following characters ‘{}();’, then the code will still won’t be able to be parsed. This is a big win.

 

Other methods make sure that performance is high is to run both the ASTAnalyser and the AutoCompleteAnalyser at the same time, the reason for this is that one of them will fail, if the code is correct then ASTAnalyser will create a new AST, but AutoCompleteAnalyser won’t work as the code is already complete. Visa-versa if the code is incomplete then the AutoCompleteAnalyser will show a pop-up and as the code is incomplete is won’t produce an AST.

 

Another method that improves performance is to use a timer which only parses the code after 200 milliseconds this ensures if the user types in a lot of text quickly it won’t bog down the IDE.

 

The final performance enhancement made was to ensure that when I am parsing a piece of code I actually run it in a separate thread, and when I try to parse something else, I automatically throw an exception to the first text that was being parsed, even if it is still parsing the text.

 

The reason for this is that if I am parsing the text again then there’s no point in parsing some old code, this is why I throw an exception even if the thread is still parsing the code. This ensures that at one time only the latest piece of code will ever be parsed.

 

All these combined gives a huge amount of performance.


8.0 Compiler/Runner interface module

8.1 Overview

 

The point of this module is to allow the user to compile or run their java work. It needs to show the text output from running any of these tasks. For example if there is an error during compilation, then the error messages should be displayed. Therefore, this module also needs to have a GUI.

 

To make the GUI more user friendly I have decided in implement the option that allows the user to be able to quickly load any java files, which have been referenced in an error message, please look at Figure 28 to demonstrate this. You will notice that I have highlighted the java files as a hyperlink, the reason for this is that users automatically will recognise blue underlined text as being a link, as a result increases the usability.

 

Figure 28 shows my compiler module compiling some code, which has error in it.

 

 

My final task of this module will be to implement the option to produce javadoc from the source code.

 

Note that there are two ways of compiling the work. One way is to compile every single file, which in the case of the project is called ‘Rebuild’, or the other option, which only compiles the files that have changed (called ‘Compile’).

 

8.2 Feasibility

 

In order for me to finish this project on time I will need not be able to write the coding for compiling, and producing javadoc, instead I will use external piece of software to do this for me. All I need to do is produce the user interface for the software.

 

This will save a lot of time. So first job I had to do was to find out which software I should use. I had many options such as makefile, gnumake, nmake, jam, and apache ant S2. However many of these tools contained flaws such as some of these tools being platform specific, others not being user friend, or in the case of apache ant being difficult to set up.

 

Now I decided to go for apache ant for the following reasons

·         Its platform specific

·         It has fantastic document support

·         It’s the most popular build tool on the market, as a result some programmer would already be aware of how to use ant

·         Ant is much more then a build tool, you can easily create scripts and extend Ant.

·         Ant will only compile the java files that have changed

 

Nevertheless, being this powerful/functional is also its flaw, as setting up a basic task as compiling requires the user to write a lengthy settings/XML file please refer to appendix C on page 69.

 

Therefore, my task will be to create a generic settings/build file which apache ANT can use to perform the tasks that I require. I could have easily implemented the other build tools into my IDE, as many of the options in ANT I will not be using. However, thinking in the long term, if I or someone else ever extended my IDE then all the other features that ANT has will become necessary and this was the point which tipped to scales towards choosing apache ant out of the other build tools.

 

Now in order for apache ant to work it requires JDK to be installed. However, in my requirement specification I stated that my IDE should work on JRE (standard java runtime) due to portability reasons. Therefore, I needed specifically find out what ant requires from the JDK, and to add that to my project.

 

Well apache ant requires the javac compiler (from the package sun.tools.javac) written by Sun. Fortunately I was able to get this library on its own, and so was able to add this to apache ant. As a result apache ant now can now work on JRE as well as JDK. (Note in order for me add the javac compiler to my IDE the user must agree to the suns licensing agreement).

 

Now in order for the user to be able to run there work I will need to run a separate JVM process, and since I am going to be doing this for the running section of this module I decided to also use apache ant as a separate process (instead of running it in my current virtual machine). The advantage of doing this is that when it comes to handling the output made by apache ant or the JVM can both be done in the same way.

8.3 Implementation

 

The way I went about handling processes was by creating an abstract class called ProcessHandler. This class basically takes in the arguments required to run a process such as ‘java –cp lib/log4j.jar;build RunJava’, and the this class runs the process.

 

Inside this class contains two abstract methods String handleErrorMessage(String message) and String handleNormalMessage(String message). All I need to do is extend ProcessHandler and write the handling methods. The reason why I went for this is that normally handling proceeses are difficult as output is given in an OutputStream, and all I want is to receive get a line of text and analysis it and update the display. Therefore, it makes sense to separate the internal workings of a process, and just have  methods which handle a line of text. The ProcessHandler can also end a process and tells you what exit value the process had. This is useful if you want to know if the process ended correctly or was there an error.

 

 

I now have a way of handling processes, the next thing I need to do is think about how to get apache ant to run in a separate process and to compiling/produce javadoc from the source code. Well this can be achieved by running the class org.apache.tools.ant.Main, with the following arguments “–buildfile <<NAME OF BUILD FILE>> <<NAME OF TASK>>.

 

Therefore, I need to first create a build/settings file. I decided to hard-code a generic build file to save time, this can be seen in appendix C on page 69. Then I just need to call any of the tasks made in the build file, such a compile, clean, or createdoc.

 

Once this is done this all that is left is to extend ProcessHandler every time I need to run the process, and to write the handleErrorMessage, and handleNormalMessage methods to they update the GUI. All this was done in a class called BuildProject. A class diagram can be seen in Figure 29 on page 42, showing the interactions between ProcessHandler, and BuildProject.

 

8.4 Testing

 

Once again, I created automated test cases, which can be found in the class ProcessTest. However due to timing reasons I made the assumption that apache ant does compile and produce javadoc correctly, and do not require to produce any automated tests.

 

9.0 External Plug-ins interface module

 

The point of this module is integrating useful features into my IDE. At the moment I have added two features successfully, this is JODE S8 which decompiles class files which is useful if the user wishes to see the java code when he/she doesn’t have the source code, and another is called PMDS12 which analyses the source code and gives productive comments on what improvements can be made. For example if your variable names are too long or short.

 

I have also been working on integrating JRefractoryS11 and CheckStyle to my IDE; JRefractory performs refractoring which is a very useful feature if you know how to use it. Unfortunately, this has not been fully integrated.

Figure 29 is a class diagram showing what classes are used to interact with any external software/plug-ins

 


10.0 JavaDoc interface module

 

10.1 Overview

 

The point of this module to display javadoc given any of the following pieces of information

·         Name of an object/class

·         Name of a method               All shown when the user selects an option in auto completion popup

·         Name of a field                  

·         Given a web address

 

The reason for this is that it firstly encourages the user to use javadoc. To save time searching through the javadoc, and to ensure transparency i.e. when the users goes through there own javadoc and or goes through the suns javadoc that they seem to interact the same way.

 

10.2 Feasibility

 

Well in order to display the javadoc I must be able to display HTML. Originally I wanted to use JEditorPane which can interpret HTML, the only problem is that JEditorPane does not comply with HTML v4 standards, and so when displaying a HTML of the javadoc, some error message are displayed on the screen.

 

However as I do not have enough time to write my own web browser I decided to use an open-source piece of software called CalpaS13 web browser to display the javadoc for me.

 

Therefore, all I need to do is tell the web browser what pages to load. You will notice in Figure 30 I have added the some toolbars. I have added back, forward button for easy navigation, and also added the ‘home’ button which loads up the users own javadoc, whereas the ‘Java API’ button loads up the main page for the suns java api. The final button ‘Current Class’ loads up the javadoc for the webpage that the user is currently editing.

 

Figure 30 shows a screen shot of the javadoc module


11.0 Server/Client module

 

11.1 Overview

 

The point of this module is to store the user’s java work onto a server, and allow multiple users to remotely connect to the server and continue working (i.e. implement remote access support).

 

The reason for this is portability, if the user can run the IDE from an applet/JNLP and is able to connect to the server. This gives the very powerful option to the user, he/she can continue working on their java work anywhere.

 

(This module wasn’t originally proposed in my initial requirement specification, as the problem of portability/remote access became only apparent when testing my software with fellow third year computer science students. Only did this requirement became visible, and since I felt that portability/remote access was an important feature in the IDE I decided to include this in my software).

 

11.2 Feasibility

 

The main functions required for this module are:

         To be able to transfer files. (Only transfer files that the user already does not have)

         Allow multiple clients to access the java code.

         Only one client can edit a specific file at one time. (Therefore ensuring that no data can be lost)

         Allow clients to communicate with each other via instant messaging.

 

From looking at the problem, it seems obvious that client/server architecture should be used, as the users need to access a central point where the java code is stored (i.e. the server). In addition, the server also needs to keep track of who is editing a file and ensure that no two people can edit the same file. (Therefore I have decided to record the state on the server instead of choosing a stateless design. The  reason for this is that having the server to record the state of each file does have the disadvantage that if the server fails then I loose the state, whereas with stateless this doesn’t happen. However stateless requires more information to be sent to and from the client which hinders bandwidth performance, and bandwidth performance is a requirement.)

 

In order to achieve fast access to the client and not to over load the server, I will temporally cache files found on the server onto the client local machine, and once the user has finished editing a specific file, it  sends the changed file back to the server.

 

The first thing I must decide is how to transfer files from the client to server, in which only the files that have changed are copied, therefore ensuring that bandwidth is kept to the bare minimum.

 

Well from hearing the words ‘transfer files’ you would automatically start thinking of using FTP (file transfer protocol), which is designed specifically for transferring files however FTP doesn’t really help when it comes to sending instant messages or ensuring only one user can edit a file at a time.

 

Then HTTP would obviously be your next choice, as you can use HTTP to transfer files, and you can add additional information like the file cannot be edited in the header. The only problem with HTTP is that firstly it is synchronous i.e. it sends a request to the server and waits until there is a response. During that time, the client cannot send anything else to the server. As a result this is not fully utilising the bandwidth therefore an asynchronous protocol would better. One final disadvantage of HTTP or FTP is that communication is one-way, i.e. that the server cannot communicate with the client until the client has made a request. This is a problem if one of the clients adds a new file to the project or sends an instant message. What should happen is that the server contacts all the other clients on-line and sends the file/message. If FTP or HTTP were used, this would not be possible (well not easily possible).

 

(Also in HTTP v1.0, once the server has sent a response, it automatically closes a connection to the client, which is not what I want it needs to be persistent).

 

However saying that it is still possible to emulate full duplex communication by ensuring the clients continuously contacts the server, even when it really does not need to send a request. That way if the server wants to send a message to the client it can.

 

It is possible to transfers information in one protocol by using another, this is called tunnelling. For example, my programme could be using FTP as a way of commutating between client and server, but I could tunnel the messages via HTTP. In fact, java servlets use HTTP tunnelling. So what this actually shows that in theory it really doesn’t matter which protocol I use as its possible to emulate that protocol by using another one, and so (nearly) all protocols have equal functionality. Saying that it may require more work if I use a particular protocol.

 

So in conclusion I decided to use my own protocol, as my problem is simple and it increases security since an external intruder would understand what information is being sent as it doesn’t comply with any specific standards.

 

The protocol that I decided to use is similar to HTTP, in which messages you have a header/key value and a result, and each message is separated by ‘&’ character. For example if I wanted to send a file the data stream would contain the following text

 

FILENAME=src/lexx/RunApp.java&HASHKEY=2D9F85A5CBAE6D08B7727FB7C41F8CC0&FILEDATA=<..content of RunApp.java ..>

 

This stream contains two pieces of information the first is that the name of the file is src/lexx/RunApp.java and the second message contains hash function of the file and the last message is the content of the file. Obviously when sending a data stream I must always ensure that if I need to send message FILEDATA it is always at the end of the data stream since the content of the file could contain ‘&’ characters, which could be mistaken as an another message.

 

As you can see my protocol is similar to HTTP, however it does not come with the restriction of it being synchronous, as no messages that I send waits for a reply. In addition, my protocol allows full duplex communication, and it does not contain any additional header information like user-agent (browser), MIME-version, Last Modified, and so on. A more detailed explanation on the headers/keys used in my protocol can be seen in Figure 31.

 

Now I need to find a way to ensure that I do not send files that the client/server already has as it wastes bandwidth. Therefore, what I really need to do is to work out what files have changed from the clients and server. A very naďve way of doing this is to compare the size of the two files, and if they are different then the file has changed. However, what if a user replaces some text then the file would not have changed in size, but the file still has changed in content.

 

An alterative would be to compare the date and time of each file and which ever has the most recent date is the modified file. However, this itself is flawed as different clients would have different times and possible even different dates on there computers. Therefore, a client have a time earlier than the server and so when it modifies a file the server would assume that the file is old.

 

An another alternative is to use a global clock which the server sets, however there are still come cases where this also fails, combining both a global clock with size of a file makes it much more reliable when it comes to finding if two files are different in content, but this is still not 100% reliable. The problem with these techniques is that they do not actually compare the content of each file and so can be inaccurate.

 

However, the only way to compare the content of each file is to actually send the file to the other user to compare which completely defeats the point of the comparing. However, a way around this to make both the server and client look at the content of the file and generate a unique hash key/function (which is 32 characters long). All the client and server need to be to send each other’s hash key and compare if they are different if so then the files content is also different.

 

However, in order for this to work I need to use a hash function that is unique. From looking at encryption, I found that the MD5 is the most common used hashing algorithm in modern encryption. In fact it is said the MD5 is 150 times more accurate then DNA itself, which as you can conclude is very unique. This is why I have decided to use MD5 algorithm to do my hashing. Also MD5 algorithm is very fast and does not use up the much of CPU resource, the only limiting factor is the time it takes to send the file (bandwidth).

 

Therefore, whenever I want to transfer a file to a client/server all I need to do is send the name of the file and its corresponding hash key. Then the server/client checks it with its own and decids if it wants the file or not. As a result, this saves a lot of bandwidth.

 

However, I can go much further, because source code is merely just text, and text has a high compression ratio. I can compress a file (by using GZip) before sending it which dramatically reduces the size of the file and so reduce the amount of bandwidth used. (Obviously the client/server on the other end with need to uncompress the file).

 

Figure 31 shows a table containing the explanation of each header/key that is used in my protocol

USERNAME

The clients username

MESSAGE 

Used in instant messaging

FILEDATA

Contains a content of a file

HASHKEY

Unique MD5 hash key of a given string or file (32 characters long or 128 bits)

FILENAME

The name of the file (including directories relative to the project directory)

REQUESTFILE

Client is requesting a files

BOOLEAN

Where a message is true of false, for example when the user requests a file

HASHPROPERTIES

Contains list all the files contained on the server/client with there corresponding hash key

CLIENT_NUM

The number of clients currently on line

SENDMEFILE

Request a specific file from the other user i.e. the client or server

SENDMEHASH

Request a ‘HashProperties’ from the other user i.e. the client or server

ENDOFFILES

Backup measure when the server has finished sending files to the client

 

 

11.3 Implementation

 

Now I have decided on the protocol and how to transfer files I now need to actually work out how to produce the client and server.

 

Well the sever creates a SocketServer (please refer to java.net.SocketServer for more information) and runs in an infinite loop waiting for an incoming connection from the client. Once this happens a new thread should be started to handle the requests for the client. Therefore my server is a multi-threaded server (thread per connection) which dramatically increases performance. However the load on each thread can be disproportionate.

 

Now the server will have two methods for sending messages. One is called sendToSpecific, (unicast) which only sends a message to a specific client (the one who is currently connected). Or the alternative is to broadcast which sends the same message too all clients who are on line. sendToSpecific method will be used to handle requests that are specific to the client for example when the client requests to edit a particular file (the answer to that should be sent to the other clients). The broadcast method is used when a client adds/edits a file, to ensure that all other clients have the most up to date information. Another case where broadcast is used is when a client sends an instant message. The server needs to relay this message to everyone.

 

In the case for the client, there is only one send method, which is called send. In the eyes of the client it is only communicating to the server, it is not aware of any other clients.

 

The client does contain two additional methods called requestFile and updateMyFiles, now these two methods are synchronous, the reason for this is quite simple. Firstly requestFile means that the client wishes to edit the file, so he/she must first find out from the server if that file has been locked or is in use. The clients needs to know the result before it can continue.

 

The updateMyFiles method is only used when the IDE is first loaded. It ensures that the user cannot load up the editor until all the files from the server have been locally cached.

 

Even though these methods are waiting for a reply from the server I can still continue receiving and sending messages as my protocol is asynchronous. Note that the server will need to store which files have been locked and by who, and any updating must also be synchronized to prevent the rare cases of two clients simultaneously locking a file. See Figure 33 to see what happens when two clients try to edit the same file.

 

Figure 32 shows a class diagram for the client/server module

 

 

 

 

 

 

11.4 Testing

 

Now testing client/server architecture is quite a difficult thing to do. Simple tests like checking my hashing, or my protocol can easily be done and automated, which can be found in these classes ChatTest, HashTest, ServerTest, and MD5Test.

 

These tests however do not check communication between client and server. In fact its very difficult to write automated test for client/server as you need to write two different automated tests and run them on different machines, which also means that I need to use two computers.

 

Therefore, the simplest way to perform communications tests is to do it manually, and for initial tests all I need to do is run the server and client on the same machine and make the ip address 127.0.0.1 (localhost). From here I can do some of the basic checks like if the files are actually being sent, is file locking actually working and to check the tolerance of the server. See Figure 33 as it shows me running two clients and a server on the same machine.

 

Figure 33 showing a screen shot of two clients connect to the same server

 

However, this is not testing if it would actually work in university nor does will it work on the network. The only way to do this is to actually run my software on two different machines, and preferably one, which is in university and another, which is outside university. This is a very difficult thing to do as I cannot be in two places at once, especially when I have changed the code and need to re-run the tests.

 

The way I got around this was to ensure that I would run just my server and client section via command line with no graphical interface, which then allows me to telnet from my home computer on to university computer, and to copy, compile and run the server. While I run the client at my home machine, this requires me to use only one computer but as the same time be able to run my work on two very separate machines. In addition, this can also check if I can go through the university firewall.

11.5 Security and Reliability

 

Whenever a programme has client/server architecture, it becomes very important to have excellent security and reliability.

 

I will first start with security, now the problem with security is that it can reduce performance and functionality therefore it is very important to get the right balance this is why I have decided to have only very simple security measures for users who connect to a server while in the computer science (CS) building as the likelihood of attack is very low, and in order for them to get access to the computers in the CS building is that they have to log in. However, if a user outside is outside university and would like to connect to a server which is running in the CS building I have implemented very high security. The only way the user can connect is by entering there own university username and password and connecting via SSH, which ensures authentication and encrypts all data that is being sent or received. As a result there is a loss in performance as some of the bandwidth is being wasted, but that is cost of having good security.

 

Figure 34 showing the login dialog for SSH connection

The reason why I can use SSH without changing my protocol is due to tunnelling. Now in order for me to implement SSH tunnelling I required to use a piece of software called J2SSHS16, which does the tunnelling for me. Note in order to SSH tunnelling to work the computer/network running the server must accept SSH connections, which fortunately Birmingham CS department does.

 

Reliability is similar to security; reliability can come with a cost. The only feature which requires 100% reliability is source code.

 

There cannot be any occasion where due to some sort of communication error the source code becomes corrupt, or deleted. This is totally unacceptable; this is why every time a file is sent the hash function for that file is also sent. This ensures two types of checks firstly the file is compressed therefore once its uncompressed the GZip algorithm does its own CRC check, once its done that then I get hash key sent to me and compare it with the uncompressed file. If they match then and only then will that file be saved in to project directory. The only way in which the data can be corrupted is if the user him/herself specifically corrupts java file and saves it. This is something which cannot be prevented as there will be no reason why a user would want to corrupt there own work.

 

Also I purposely restricted the feature that no client can delete a file from the server, they can only edit or add files to the server. That way in the unlikely event of an attack the damaged caused is kept to the minimum.

 

Now if a file is being sent and for some strange reason the client/server suddenly disconnects or crashes the source code cannot be corrupted as files are only saved when message has been fully sent and checked twice.

 

You may be asking yourself what happens if the client suddenly disconnects? Well as the connection to the server is persistent, the connection will be dropped and the server unlocks and files that was locked by that user. On the other had what if the client crashes? Well the connection is still there, so I have implemented a timeout, in which if the client does not communicate with a specific amount of time the server automatically disconnects it and unlocks any locked files.

 

What about all the work that the client was editing that wasn’t saved/sent to server before crash/disconnecting?, well unfortunately that work will be lost. The reason for this is that I cannot implement the option when the client can reconnect to the server a force the server to accept that clients java work as it can create inconsistencies. For example let us say we had five clients all working on some java work, one client suddenly get disconnected and the other four continue working. Now if I allowed that single client to copy his/her work back onto the server then all the work that the other four clients were doing would have been lost. Which is even worse, this is why I cannot allow this to happen; it is the best option out of a bad bunch.

 

What about the other hand what happens if the server disconnects? Well the connection will be dropped and the client will automatically show a pop-up telling you that you are now longer on the server, and you need to reconnect or restart the server. Once again, if the server crashes I implement a timeout feature to detect this.

 

If the client crashed/disconnects the server handles this very comfortably, however if the server crashes then the client cannot continue doing there work, which is very serious. This is a common problem with any client/server architecture; the only way around this is to use backup servers, which come into play when the primary server goes down for some reason. However due to my strict schedule time I will not be able to implement this, but it is a recommendation for future development.

 

What about tolerance? How many connections can the server handle, well if all the clients are sending and receiving information from the server continuously then usually after ten clients the server starts to throw exceptions by either dropping or refusing connections. If however you have clients, who send messages rarely then the server can handle much more connections from 30-40 clients.

 

So I would say the server can comfortably handle eight clients irrespective of the load, however to be on the safe side I would recommend up to six is the maximum number of clients that can be simultaneously be connected to server.

11.6 Portability

 

In order for remote access to be portable (in the sense that it runs on more machines) it would be advantageous to use a couple of different protocols. As you have already read due to tunnelling this is very easy thing to do. At the moment I use my own protocol which requires a socket and communication is direct. Also you can communicate via SSH for a more secure connection.

 

However for very restrictive firewalls, usually only allow incoming and outgoing HTTP connections (v1.0) are accepted. As I already said, I could perform HTTP tunnelling, and include full duplex communication in HTTP v1.0 I ensure that the client always sends a dummy message to the server. (This will require me to change some the code of the client and server to accommodate for this).

 

I was unable to find any external software to perform HTTP tunnelling for me, so I had to write this out myself. First things first I need to have a http server to accept incoming HTTP connections, my first option was to use tomcat, and make my server into a servlet. However setting up tomcat and converting my server into a servlet does require a lot of working and setting up, and tomcat is really too powerful for my purposes.

 

All I need is a simple HTTP server, fortunately the suns developer website has a technical articleS14 on how to write your own HTTP web server, and it comes with a very basic web server example. However, this example is very simple to understand and easy to implement into my code. This is why I decided to take this code an added it to my project. (Note have put it in its own package example, and lexx.server.http packages has all the code what was written completely by myself regarding http tunnelling.)

 

So my programme can use three protocols a direct socket connection, SSH, and HTTP. However the HTTP connection is terrible when it comes to bandwidth as dummy messages are being sent inorder to allow full duplex communication and as HTTP is synchronous the client cannot send and receive at the same time.  Also it not very good at detecting if the client has disconnect or crashed, only a timeout can do this.

 

HTTP should only be used if none of the other protocols is accepted. For the purpose of Birmingham University HTTP will never be used, as you can always use SSH or direct socket connection. I only implemented HTTP for completeness, and portability reasons.

 

11.7 Performance

 

Well direct socket connection is the fastest, then SSH and finally HTTP. The reasons why my client/server has high performance is due to

:           Only files that are new or have been changed are transferred (works very

:           All files are compressed before transfer (on average java files are compressed to one third of its original size)

:           Only files that are new or have been changed are transferred (dramatically saves time if you are using a computer which has connected to the server in the past).

:           Server is multi-threaded and so can send and receive messages to all clients simultaneously

:           Asynchronous protocol is used and so client can be sending a message at the same time as receiving a different message from the server. Full utilisation of bandwidth

:           All files that are going to be sent are loaded into memory first, since if I directly read the file from the storage medium (e.g. hard disk) and you have many clients requesting files then it can put a big strain on the storage medium and actually start to slow down transfer. Also if prevent the server from writing to an existing file.

:           For my own protocol (using direct socket connections) I use UDP instead of TCP. UDP is faster than TCP but less reliable, and packets can be sent in the wrong order. However say that both the client/server performs many of its own checks regarding a message, there is no need to add the additional overhead of TCP. Only thing I need to do is to perform marshalling and unmarshalling which is easy to do.

 

So even if a client had a 57.6 kbps connection and working on a large project for example mine he/she could still continuing working without any waiting. For example at the moment my project is about 13.2Mb. However majority of that are test cases, images, and libraries, which will never change. Therefore the will be a high one off cost to initially download these files, but once done will never need to be downloaded again.

 

Now lets say that the client has downloaded everything once, and he/she exits. Then another client connects to server and makes some changes let us say changes 6 java files, which combined is 100Kb in size. The next time the client with 57.6 kbps connection comes in, he/she would only need to download the changed files and due to compression, he/she would only need to wait until 33Kb have been received (which is about 5 seconds).

 

Figure 33 show an example of this. There is one client which hasn’t cached any files, and another where the files have already need cached you’ll notice what one client has received over 8Mbs whereas the cached client only received 0.77Mbs.

 

11.8 Integration & User Interface

 

The client/server module is transparent as it sends and receives files independently to the IDE/editor. However there a only a few occasions when the IDE/editor needs to interact with the client. Firstly, when the user wishes to load a java file up, the editor asks the client first or when the user closes the java file (unlocks it). The last case is when the user successfully compiles the source code; all the changed files are sent to the server. This last case is not really a requirement it merely ensures that the clients have the most up to date java code which is complete.

 

For the instant messaging all that the editor does is adds a panel, which is given by the client, it doesn’t do anything else. Please refer to Figure 33 and Figure 37 to see an example of instant messaging.

 

 

 

 

 

 

 

Figure 35 shows a screenshot of files being locally cached onto clients machine

Figure 36 showing a dialog for connecting to a server

Figure 37 showing a screen shot of instant messaging

Figure 38 shows a screen shot of the types of messages what the server sends and receives

 

 

 

 

 

 


12.0 Testing

 

12.1 Automated Testing - JUnit S1

 

Wherever possible in the project I have tried to use piece software called JUnitS1, which is primarily used for unit testing. The advantage of using JUnit is that it takes test cases/test suites that I have written and runs each test, showing me if any of them have failed.

 

Figure 39 shows JUnit running a test case

Having automated test cases allows me to runs these tests as many times as I like. This is good when I am modifying my code, as these tests can check if these modifications have not corrupted the system/module. This becomes even more important if maintenance is required, and someone else if editing the code.

 

If I manually did testing myself, I would not be able to do testing throughout my project as it takes a lot of my time up, but with automated tests the computers does all the hard work. In fact in (eXtreme programming) XP software methodology requires all tests to be automated.

 

Writing a test case is very simple, all I need to do is to create class that extends TestCase, and have static methods with the name test as the beginning. Inside these methods, I need to insert the statement assertTrue with the boolean to say if the test was a successful or not.

12.2 Logging - Log4J S7

 

Log4J is a logger, which becomes vital once a test case has failed. As you use the logger to find exactly where the defective code lies.

 

A logger is similar to System.out, or System.err command. You merely put some text describing what is happening. The advantages of the logger compared to System.out are:

P       Messages are classified in groups i.e. fine, info, debug, warn, and fatal.

P       You can find out which object sent the message

P       There is the option to filter classes, so the message from certain objects are only shown or only messages that are warnings or fatal can be shown.

P       The logger can send messages/log to nearly anywhere. It can be sent to a file, or displayed on screen or sent over the Internet.

P       However most importantly you can turn off logging to ensure that there is not any loss in performance, as all the log statements will be ignored. (You may need to add an if statement like if( log.isDebug() ) before some log statements)

 

The problem that I faced at the start of the project is that I would put these System.out statements in my code, to find and fix any bugs. However once I fixed the bug, I would get rid of these statements as they slowed down the system/software, but later on in the project, I would get another bug and so put back these System.out statements.

 

This why I found logging to be valuable tool in testing/debugging as it allows me to keep these log statements permanently in my code. All I need to do is to tell the logger to ignore all log statements. In fact, I was so impressed with logging that I decided to add it to my project even though it was not originally in my specification. This was done because I felt logging was so useful to a programmer.

 

There are two main loggers in Java, log4j by Apache and a new logger by Sun, which has been added to JDK 1.4. The reason why I chose Apache lo4j instead of the Suns one, was on the simple fact that the Apache logger has been on the market much longer then Sun, and so I found that there was better support and documentation for it.

 

Figure 40 shows a screen shot of logging in my IDE

12.3 Visual Logging - Graphical Components in Log4J

 

Now logging is not specifically designed to store graphical components but instead to store text/string messages. In my project, I used data structures (AST and Trie) which have a tree like structure. Now when debugging I need to visually see these trees, to understand if the content is correct. Displaying them in a string/text format looks confusing. Therefore, required the logger to store the state of these data structures, which could be displayed visually i.e. in a JTree (javax.swing.JTree). As a result I had to find away of converting the components into a string.

 

The way I solved this was to ensure that my data structures were Serializable and Clonable and that they were stored in a JTree. Then I could use serialization to make a copy of these objects which could be sent to the logger.

 

However the logger only accepts string messages, therefore I must convert the serialized object into a text /string. This is where the XMLizableS9 comes in, XMLlizable is a piece of software which converts serializable objects into XML code, and back.

 

The tool that I am using to display the log information is an open-source programme called LumberMillS6. As its open-source I was able to modify the section of code which interprets the log messages, and add a piece of coding to check if the message contains any XML statements. If it does it then converts the XML message into a component.

 

This is not a stand way of doing logging, however I required a visual way of showing of data structure.

 

12.4 Simulator

 

Unit testing is very useful for testing individual modules, however it does not help when it comes to integration and system testing, and it is very difficult to test functions, which are graphical. This is why I decided to write my own simulator that can solves these problems, and produce automated test cases.

 

The way this works use the given the name of a zip file found in the lexx.test package to the AllTests class which automatically unzips the file into a temporary directory. Then setups all the required classes such as JavaPackages, EditorFrame, EditorPanel. Once I’ve done that I can use methods in the Simulator class which enters text into the editor, by telling the Simulator what text you wish to insert and at what position. Instead of just inserting the whole text, it actually inserts the text letter by letter as if someone was typing it. (There is one fifth of second delay between each letter which is the average speed of typing). If a popup is displayed the Simulator will get all the components in the popup and actually reads the text of each component. From this I can tell the Simulator if a certain method/field exists. (This is why if you run lexx.test.AllTests which is all my automated test cases, you will notice that sometimes a popup will appear for a split second, this is when I am running the Simulator class and it is entering text into the editor.

 

I can also use the Simulator to compile code and get the output, produce javadoc, and even check if my autocompletion also takes into account inheritance. With the Simulator I can perform a huge range of tests, and each individual test starts at the beginning i.e. reading in the java runtime to actually editing/saving/compiling the code.

 

However there are some disadvantages in using the Simulator firstly if it fails a test it gives very little information where it has failed as it uses a wide range of modules. It also takes a long time to run a test, as it has to unzip the file, then read in the java runtime, enter text letter by letter. This can take up to 15 to 30 seconds to perform a single test, which may not seem much. However as I can perform a wide range of different tests I can produce a large number of test cases, and so this time do start to add up. With my unit tests, I can run all of them and it would be finished in about 20 seconds.

 

Now it takes about 5-6 minutes to run my test cases that use Simulator class.

12.5 Using my IDE to write my IDE

 

This is somewhat of a strange way of testing, but once my editor was at a specific standard, I was actually using my own IDE to write more code to the IDE. This is not a very controlled way of testing it is more like field or end-user testing. (It is an indirect why of doing testing), and it actually can checks if it performs my primary objective i.e. to help java programmer to write programmes easier.

12.6 Computer Science Students

 

At the starting, middle and ending of the project I used some third year students to get critic of my IDE. At the beginning they were used find requirements what they thought would make a good java IDE.

 

In the middle and end a few students actually used my software and got feedback of what they thought about it. (Mid-way in the project the remote access feature was discovered by response from the students). However the main pointers I got from the students were regarding the user interface on how they though it could be improved.

 


13.0 Analysis

 

In this chapter, I will be evaluating the project as a whole, did it meet the requirements, how does it compare to other software of the similar type.

13.1 How the IDE meet its requirements

 

If you mean how does my IDE compare to my initial requirement specification then it easily exceeds it as I have also included remote access support. If you mean my updated requirement specification then generally it was all met.

 

Figure 42 shows a table giving a critic on how each section met its requirement

Requirement

Comments

Text Editor

Complete – incorporated cut/copy/paste and the undo/redo features. It allow the user to load and save their work.

Auto Completion

Complete – the methods/fields of a given object/class is shown as the user types. My initial requirements were only looking for basic auto completion whereas my current version is fairly advance as it can take into account inheritance, and it is very fast and meets the performance requirements.

 

Saying that it has weak support for arrays (but my requirements where suppose to support arrays), and it doesn’t work well if the code is messed up (unbalanced braces), but then again that the same problem that other IDE’s have such as JBuilder and Netbeans.

Syntax Highlighting

Complete – Currently syntax highlighting can be done in two ways. First one is to use syntax highlighter which I wrote from scratch via StyledDocument, however due to poor performance of StyledDocument, so I went for another way which was to integrated a piece of software called SATC. SATC performs syntax highlighting on ANTLR grammar. It is much faster then mine but not that good when the code is incomplete. However due to SATC I can now show syntax highlighting for HTML/XML, and ANTLR which is not even in my requirements.

Compiling / Running

Complete – Even from my initial requirements I had already decided that I was going use Apache Ant to perform these tasks, and all I needed to do it produce the GUI and control the output, which my IDE does.

Remote Access

(Client / Server)

Complete – Users can connect from there home computers to university even with a slow connection.

Portability

Complete – Users can run my IDE from a website requiring only JDK 1.4 or above via an applet or JNLP.

Producing or Viewing JavaDoc

Complete – Users can create java doc via apache ant and can view it via Calpa web browser. It also views javadoc for the methods and fields displayed in auto completion.

External Components

Complete – However the requirements were never specific in this section but I have included some useful features such as PMD which analyses the users code.

13.2 GUI and Usability

 

This is somewhat of a difficult requirement to evaluate. As there is usually a conflict between usability and functionality. Therefore, it is very important to get the right balance. In this project I ensured all the main functions could be achieved within 3 clicks and at the same time I included shortcut keys for advance users who wish to perform tasks quickly.

 

I included mini wizards to help the novice programmer to create classes and to add files to there project. I allowed the user to have much more flexibility in choosing the look and feel of the software via L2FprodS10.

 

I allowed the user to able to quickly switch from different java/text files, or view javadoc within 1-2 clicks. I added tooltips to nearly all buttons so the novice user can get more detail on what certain things can do.

 

Adding simple features like remembering which projects were lasted entered, or what was the username or ip address that a client used to connect to server and so on. All this make the software just that bit more user friendly.

 

Evaluating GUI is less obvious, I have applied many of the concepts in HCI (Predictability, Synthesizability, Familiarity, Generalizability, Consistency, Dialogue initiative, Recoverability, and Multi-threading), but that does not necessarily guarantee a good GUI, however I did get useful feedback from third year students on what they thought about the GUI.

 

If I take their thoughts on then the GUI is quite good, however I do believe there is still much room for improvement. Such as

C                   More wizards - to help the novice user build stubs for the java files

C                   A help file - which gives a detailed support when a user is unable to understand what a function does.

C                   Customisation –At the moment this is limited. For example the user cannot change the colours in syntax highlighting.

 

Now the reason for these features not being in my current GUI is because they are somewhat trivial tasks to do, but take a huge amount of time to complete, and as my time schedule is very tight, I made the decision not to include these features. However, these can easily be added little difficulty since I have not hard coded the colours for syntax highlighting they are actually stored in a configuration file so all the programmer needs to do is modify that file.

13.2 Reliability

 

In my current project reliability can be split into 3 sections: GUI, auto completion, and remote support. As each section has a different requirement for reliability.

 

For GUI – Good reliability means that the user can easily change there mind anywhere in the GUI. For example the user chooses to create a java file, and half way in filling in the details suddenly changed his/her mind, then the user should have the option to go back the editor screen. This is what I have been trying to do nearly everywhere in my java programme. The user does not need to commit to anything he/she can always go back. Therefore reliability for GUI is good.

 

For auto completion – reliability is average. As detecting objects/variable, declarations are very sensitive to the java code. If the braces do not match then it cannot detect anything. Fortunately, my code remembers object/variable declarations previously when the code was correct. However any newly entered text may not be included. This is actually very bad, but to my defence this is a common problem with other IDE’s which have auto completion. They will also fail if the code is incorrect.

 

Saying that there is one advantage of auto completion, which mine can do and other IDE’s with auto completion cannot do. This is class auto completion. I am referring to something like ‘System.’, as this is a class not an object. Therefore I do not need to analyse the java code, and so I can do auto completion for classes when the java code is incorrect, but I cannot do it for objects.

 

Strangely enough other IDE’s like JBuilder and Forte still won’t do auto completion for classes when the code is incorrect. This is why I have given reliability to average, as auto completion isn’t that reliable but compared to other IDE’s it is still better.

Finally, for remote access – reliability is good. As there is a lot of error checking and the sever can handles very well if a message is corrupt or when the client crashes or disconnects. It can handle six clients very well, but can go ten clients and still work even if there is a high load. The server ensures consistency when it comes to file locking. The only problem with remote access is if the server fails, the way around this is to implement a backup server but this was due to scheduling issues this was not possible.

13.3 Performance

 

Generally threads will always increase performance aslong as there is not too threads (as they have a overhead) and there isn’t too much I/O (as seek time is very slow). Where ever possible I have tried to user threads in my software. For example the user can still continue editing some code even when it compiling / producing javadoc. (Other IDE’s like JBuilder or Netbeans do not allow this to happen).

 

Once again, performance can be split into three sections bandwidth (via remote access), timing (via auto completion), code size (via source code of IDE).

 

For bandwidth – performance is excellent due to compression, hashing and caching. Once again, performance can be split into three sections bandwidth (via remote access), timing (via auto completion), code size (via source code of IDE).

 

For timing – performance is good. If you are comparing it to my requirements in which auto completion must occur within one second then performance is excellent, however when you compare it to other IDE’s especially JBuilder my auto completion is not as fast as there’s.

 

However if the java code is very large for example 150Kb then auto completion can exceed one second, but it is very unlikely if the user would ever be editing a file that is that big, and if you use my incremental parser (which isn’t working 100%) then I can still easily achieve under one second.

 

The last test for performance is code size. On average if you have two programmes which perform the same tasks, and one is half the size of the other, then you would normally say the smaller sized programme is better.

 

The size of a programme is useless without comparing it to another programme which performs the same task. First thing first you cannot directly compare the size of the source code as in java you put javadoc and comment into your source code. Therefore a programme with poor documentation will always be smaller than a programme with excellent documentation.

 

So the way you compare java programmes is the size of the class files as they don’t include documentation. Now my programme is 10Mb big, however this is a very misleading as it contains which are not actually part of the IDE. For example it include the JUnit libraries which is only required for testing, it includes JRefractoryS11 and CheckStyle libraries which are not even in my final version as I am still looking at these programmes but haven’t fully integrated it with other programmes. Also that 10Mb include images, and test cases with zip files containing source code, and also ANTLR which is only required for generating a java parser.

 

In fact if you only include the classes and images that are actually used in my software the size becomes 3Mb, and if you remove all the look and feel skins then the size becomes 2.3Mb, and note that this includes Apache Ant, Log4J and the suns compiler. Therefore 2.3Mb for an IDE with auto-completion , compiling, and remote access support is very good.

 

However comparing it with other IDE’s is difficult as programmes like JBuilder and Forte come with additional features which make their size huge (40 to >100Mb), and the IDE’s which are similar size to mine don’t come with the features that I have.

 

The only piece of software that I could find which had similar features to mine is a programme called jExt (http://www.jext.org/ ) it has the same features as mine with the exception of remote access and portability. Its size is 3.8Mb and mine is 3.0Mb.

 

 

I can compare programmes, which are just text editors with syntax highlighting support, as there is a lot programmes out there that do this task. Now the way my source code is organised in distinct packages and so it is very easy to workout what the size of my programme would be for just the text editor with syntax highlighting. (This was not achieved my planning this at the starting, but my continuously using refracting to organise and simplify my code).

 

 

 

 

 

Figure 43 showing table comparing the size of different text editors

Programme

Size – (class files)

Jipes- http://www.users.globalnet.co.uk/~eis/jipe.htm

502Kb

XEmacs (Java Developer Edition)

1600Kb

jEdit – http://www.jedit.org/

1400Kb

Text editor - http://www.studioware.com/

1851Kb

My IDE with just text editor and syntax highlighting via SATC

521Kb

My IDE with just text editor and syntax highlighting via my old algorithm

376Kb

 

Well as you can see in Figure 43 and from my previous explanations the size of my program is actually quite good considering the size of my programme.

13.4 Portability

 

My second objective was to produce a portable IDE which allows the user to run the IDE straight off a website and remotely connect to a server to continue working on there code.

 

Now this was successfully achieved by ensuring that java runtime was only required not the java development kit. This meant that I had to include the suns java compiler as one of my libraries; it also meant that my IDE doesn’t require any installation but at the same time is able to store configuration information if the programmer uses the same computer.

 

In order for my IDE to run off a website there are two options.

·         Applet – a traditional way of running programmes from web pages, however if the user closes down the web page then the programme will automatically quits. There is no way of preventing this from happening this is my I recommend programmers to use JNLP/java webstart instead

·         JNLP (Java Network Launching Protocol) – This was specifically designed to run applications from websites without actually running it from a web browser. It automatically creates desktop icon for the IDE and only downloads the IDE if it has changed. The only problem with JNLP is that not all people have the programme called ‘Java WebStart’ or OpenJNLP which is required for it to work. (For example if you are using the Sun computers in Birmingham University). This is why I use the backup of applets.

 

Obviously in order for programme to run via the website I need to sign the jar files and ask the user if I can have access to there hard disk in order to load and save the java files.

 

The remote access support has been working excellently and so portability has been fully achieved. You can run the IDE from this website http://lexx.sourceforge.net . When I originally proposed my project, I could not find any IDE which was portable at the moment I believe mine is the only one.

13.5 Cost

 

The main cost of this project was my own personal cost. There was no finanicial cost as all software used where open source with the exception of suns java compiler which is free if you accept there licencing agreement.

13.5.1 Personal Cost

 

The personal cost of this project was very high. This is very apparent from just looking at the amount of source code I have 906Kb was completely hand written by myself, and the additional 352Kb was either modifying existing code or generated by ANTLR.

 

In addition, large amount of research was required in order to learn how to use ANTLR, ANT, JEditorPane and SocketServer/Socket.

 

All this added up gives a high personal cost, and for a third year project it is a bit too much work, however I was aware of this at the beginning of the project, and since I chose this project myself and felt very strongly about it I was very happy to put this extra work in.


14 Conclusion

 

This project was a very demanding which required a huge range of different skills, as the current moment I have built a portable java IDE with the some very useful features, which a java programmer would like. However there can always be room for improvement. I believe I have built a good enough framework so it can be easily extended.

 

I am very proud of this project and the only regret I have with the way I went about doing this project was at the beginning where I changed the auto completion algorithm to ANTLR without doing in-depth research of why my algorithm was so slow and only finding later on that with wasn’t anything to do with my algorithm but more to the fact that I was using StyledDocument. However saying that using ANTLR and SATC has made the design much more generic and so it can easily be extended.

 

Now since my project was initiated due to my quarrels with XEmacs it makes sense that I should compare my IDE directly with XEMacs, and see how it competes with mainstream IDE’s like JBuilder and Forte/Netbeans. This can be seen in Figure 44. Note that all comparision made with software that were available when project was proposed.

 

14.1 Possible Extensions

 

The possible extensions are

·         Writing a backup server just in case the primary server ever fails

·         Adding more wizards to GUI to help the novice user

·         Add configuration dialogue so user can change colour of the GUI

·         Add help file

·         Add the option to delete and rename files

·         Finish the integration of JREfractory so users can perform refractoring

·         Integrate JUnit to IDE

·         Include a debugger and profiler

·         With my generic/abstract auto completion classes, and using SATC users can write auto completion and syntax highlighting for other OO languages (and possibly some non-OO languages).

·         Add printing support

·         Implement a GUI builder

 

The possible extensions are limitless, however I would recommend the first six extensions to be made.


Figure 44 shows a table comparing my IDE with XEmacs, JBuilder and Netbeans

Feature

Lexx IDE

XEmacs

JBuilder

Netbeans

Text Editor

Good

Good

Good

Good

Syntax Highlighting in Java

Good - Fast, but doesn’t update syntax straight away. (Such as java doc is only highlighted once user types ‘*/’)

Excellent

Excellent

Excellent

Syntax Highlighting in other languages

Good

Excellent

Good

Good

Code Beautifier

Poor – only the most basic code beautifier

Excellent

Excellent

Good

Auto completion on objects

Average – Does not work if code has incorrect syntax.

Unable to perform function

Good – Does not work if code has incorrect syntax. But is fast

Good – Does not work if code has incorrect syntax. But is faster then Lexx IDE

Auto completion on classes

Excellent – As it works even if code is incomplete

Unable to perform function

Average – Does not work if code has incorrect syntax.

Average – Does not work if code has incorrect syntax.

Auto completion with inheritance

Average – Only works if code has been compiled

Unable to perform function

Good – works even if code isn’t compiled

Good – works even if code isn’t compiled

Auto completion while showing JavaDoc

Excellent – Shows the javadoc at bottom of pane

Unable to perform function

Unable to perform function

Good – Only shows the javadoc when user is selecting a method/field

Producing JavaDoc

Good

Unable to perform function

Good

Good

Viewing JavaDoc

Good – inbuilt web browser

Unable to perform function

Unable to perform function

Average - Only shows javadoc for methods/fields

Compiler

Good

Poor – doesn’t compile all code

Good

Good

Running java programme

Average – doesn’t take keyboard input

Poor – as it can run classes that are in packages or require external classes

Good

Good

Integrate Logger

Excellent comes with Log4J and uses LumberMill GUI to display the logging information

Unable to perform function

Unable to perform function

Unable to perform function

Debugger

Unable to perform function

Unable to perform function

Good

Good

Size

Excellent - 3.0 Mb

Excellent – 1.6 Mb

Poor - > 100 Mb (even higher if you include documentation, extras)

Good – 40Mb

Platform Dependence

Excellent – Platform independent

Poor – specific to each platform

Good-Poor – Is written in java, but there are different versions depending on platform

Good – There is a platform independent version but requires many additional libraries which have to be downloaded separately.

System Requirements

Average – requires average spec machine

Excellent – works on very low spec machines

Poor – requires high spec machines

Poor – requires high spec machines

User Interface

Average – Excellent UI but lacks help file and customization

Poor – common complaint by third year students

Average – Excellent UI for advance users but can be overwhelming to a novice.

Average – Excellent UI for advance users but can be overwhelming to a novice.

Java Requirements

Excellent – works with java runtime 1.4+

Average – Uses java development kit

Average – Uses java development kit

Average – Uses java development kit

Portability

Excellent – can run via website

Poor

Poor

Poor

Remote Access Support

Excellent – works efficiently with low bandwidth

Unable to perform function

Unable to perform function

Unable to perform function

 

14.2 Lessons Learnt

 

I have learnt a lot from doing this project specially in the programming side, and understanding the problems faced in compilers and in large projects.

 

However, the main things which I am very proud of in this project and I believe have taught me invaluable lessons are refractoring, logging and unit testing. Mastering rerfactoring , and unit testing are skills which I did not have nor was I taught, but can dramatically increase the quality of a programme.

 

Refractoring allowed me to simplify my code, and to continuously scrutinise my design throughout the development of the software. I found that it doesn’t matter how much planning and designing you can do you will always hit an unforeseeable obstacle which requires your coding to be slightly changed (irrespective of how modular your code is). Refractoring allowed me to change the design of the code without changing its semantics. In fact there is a package called lexx.utils which contains classes which are all static factory classes that were produced via refractoring.

 

I found that where was a some section of coding which I was repeatedly using, and so I merely selected the region of code and moved it into a static method for all other classes to do. I also used refractoring to recovering nearly all my code from the migration from the initial auto completion algorithm to my latest one (which uses ANTLR). I believe if I did not use refractoring then majority of the code would have been wasted.

 

Unit/automated testing is a skill, you can’t just add it at the end of implementation. You need to actually think how you can you automate tests and what changes are required for this to happen. In fact, now I cannot imagine programming without first thing how can I write an automated test case for it. Manual testing is something that is boring and always left at the end, whereas automated testing is best done throughout the development of the project.

 

Logging is very useful in debugging, and is a simple skill to learn.

 

Refractoring, unit testing and logging all improve the quality of the software however it isn’t actually seen at the end-product, which is unfortunate as this is a reason why many programmer are unaware of this.


External Software/Tools used in Project

 

S1 JUnit  - http://www.junit.org/index.htm

Used to create automated test cases for Testing phase of development

 

S2 Apache Ant - http://ant.apache.org/

Build Tool – used in my IDE to perform compiling and to produce javadoc

 

S3 Suns Javac Compiler - http://java.sun.com/j2se/

Required for Apache Ant in order to compile code. This library is included in JDK, but not in JRE (java run time).

 

S4 Find & Replace Text Package  - http://www.calcom.de/eng/dev/cctxt.htm

Used in my IDE to do find and replace in a textpane.

 

S5 Syntax Aware Text Components – by Bogdan Mitu

Used to perform incremental syntax highlighting on ANTLR lexers/parsers

 

S6 LumberMill  - http://traxel.com/lumbermill/ 

Used in my IDE to have a graphical logger

 

S7 Apache Log4j  - http://NEED TO FINDOUT log4j.sshtools.com/j2ssh.html

Used in my IDE for logging purposes

 

S8 Jode  - http://jode.sourceforge.net/ 

Class Decompiler – Used in my IDE when source code is not available

 

S9 XMLizable  - http://iharder.sourceforge.net/xmlizable/

Converts serilization objects to XML code so it can be transported in text format to LumberMill

 

S10 L2FProd  - http://www.l2fprod.com/ 

Used inside my IDE to allow my GUI to be skinnable.

 

S11 JRefractory  - http://jrefractory.sourceforge.net/ 

Used inside my IDE to allow refractoring support

 

S12 PMD  - http://sourceforge.net/projects/pmd/

Used inside my IDE to analyse source code to check for any design errors

 

S13 Calpa - http://www.junit.org/index.htm

Web browser – Used to show HTML web pages, which is required to show javadoc in my IDE.

 

S14 Suns Basic Web Server - http://developer.java.sun.com/developer/technicalArticles/InnerWorkings/JDCPerformTips/index.html

Basic HTTP Web Server – Used in my IDE to allow HTTP protocol to be used.

 

S15 ANTLRhttp://www.antlr.org 

Parser Generator – Used in my IDE to produce a java parser

 

S16 J2SHH  - http://www.sshtools.com/j2ssh.html

Used in my IDE to perform SSH tunnelling.

 


Bibliography

1 Sommerville Software Engineering Book, 6th Edition

Details regarding software design pages 56-58

Architectural design – pages 216-236

 

2 http://java.sun.com/products/javawebstart/1.2/docs/developersguide.html

2 http://www.vamphq.com/jwsfaq.html

2 http://openjnlp.nanode.org/

How to use JNLP

 

3 Using Java 2 Enterprise Edition - Special Edition, by Mark Wutka, QUE

Reflection – pages 747-754

 

4 http://www.cp.eng.chula.ac.th/~somchai/2110211/Java/Collection/Chapter2-CoreJava2/page6.html   

Chapter 2: Collections, Bulk Operations – Used to show how to perform sorting, and how to use the properties class

 

5 http://www.javaworld.com/javaworld/javatips/jw-javatip104.html

Make a splash with Swing

 

6 Core Java 2 Volume 2

Help on JList pages 410-411

 

7  http://javaalmanac.com/egs/java.util.regex/ReadPara.html  

Using a Regular Expression

 

8 http://java.sun.com/docs/books/tutorial/uiswing/components/tree.html

How to use trees

 

9 http://www.ca-osi.com/article.php?sid=65

MD5 Hashing in java

 

10 http://research.sun.com/people/mlvdv/COM.Sun.Labs.Forest.doc.coset_2000.paper_pdf.pdf  

Displaying and Editing Source Code in Software Engineering Environments

 

11 http://www.is.titech.ac.jp/~sassa/papers-written/nakai-sassa-waga00.pdf

Incremental Attribute Evaluation of LR-attributed grammars

 

12 http://www.mokabyte.it/2000/06/firewallutil.htm

Let Java Applications Through Firewall – (Tunnelling)

 

13 http://www.wakhok.ac.jp/~tatsuo/sen97/10shuu/GZip.java.html

Example of how to use GZip in java

 

14 http://developer.java.sun.com/developer/technicalArticles/InnerWorkings/JDCPerformTips/index.html

JDC Performance Tips and Firewall Tunnelling Techniques

 

15 http://java.sun.com/j2se/1.4/docs/guide/net/properties.html

Networking Properties

 

16 http://java.sun.com/docs/books/tutorial/networking/sockets/clientServer.html

Writing a server side of a socket

 

17 http://www.developer.com/tech/article.php/960501 

Writing a one-many chat

 

 

18 http://www.jguru.com/faq/view.jsp?EID=585187 

How to use javac in ANT

 

 

 

19 http://java.sun.com/webservices/docs/1.0/tutorial/doc/GettingStarted5.html 

Building the Getting Started Application Using Ant

 

 

 

19 http://www.w3.org/Protocols  

Discusses the HTTP protocol

 

Appendix

 

Appendix A- Architectural Design

 

Figure A.1 showing the architectural design of my project. Indicating the core modules, and their type of interactions.

 

Figure A.1 doesn’t include interaction with external components like Apache Ant. This however can be seen in figure A.2. Note that the bold red text indicates external software components.

 

Figure A.2 showing the complete architectural design of my project.

 

Appendix B- Initial Prototype

<<need to do>>

 

Advantages

·                 Very good tolerance for code having incorrect syntax

·                 Simple

 

Disadvantages

·                                         No error checking to find if code is syntactically correct.

·                                         No efficient way of updating the analyser when code is changed

·                                        Very slow ( however this can could have been optimized further)

 

Figure 45 is a screen shot of my proto-type. It showa how the java text is broken down into a tree.

Appendix C - Generic Build File for Apache ANT

 

<project name="Lexx" default="dist" basedir="C:\Java WorkShop\Lexx">

<description>Automated build file</description>

<property name="src" location="src" />

<property name="build" location="build" />

<property name="dist" location="dist" />

<target name="init">

  <tstamp />

  <mkdir dir="${build}" />

</target>

<target name="compile" depends="init" description="compile the source">

<javac srcdir="${src}" destdir="${build}" listfiles="true" deprecation="on" debug="on">

<classpath>

<pathelement path="${classpath}" />

<fileset dir="lib">

<include name="**/*.jar" />

<include name="**/*.class" />

 </fileset>

</classpath>

</javac>

<copy todir="${build}">

<fileset dir="src">

<exclude name="**/*.java" />

</fileset>

</copy>

</target>

<target name="dist" depends="compile" description="generate the distribution">

  <mkdir dir="${dist}/lib" />

  <jar jarfile="${dist}/lib/MyProject-${DSTAMP}.jar" basedir="${build}" />

</target>

<target name="clean" description="clean up">

<delete dir="${build}" />

<delete dir="${dist}" />

  </target>

<target name="createdoc">

<javadoc packagenames="*" sourcepath="src" destdir="docs" author="true" version="true" use="true">

<classpath>

<pathelement path="${classpath}" />

<fileset dir="lib">

<include name="**/*.jar" />

<include name="**/*.class" />

</fileset>

</classpath>

</javadoc>

</target>

</project>