Debugging Trick in Eclipse

Introduction

Today a friend asked me if you could execute an action in Eclipse on a breakpoint. I thought for sure that that would work, but decided I had to check before I said yes. Well… there is no such feature in Eclipse, but here is a cool work-around.

Problem

The problem that my friend had was that he’d like to inject some code when the breakpoint triggered, print out some stuff, then resume operation as quickly as possible. The reasons for this may be many, but the most typical is that you’re running in some real-time (or semi-real-time) scenario where stopping in the debugger may make you miss a deadline.

Solution

Create a specialized break in Eclipse that has the following traits:

• Make the breakpoint conditional
• Specify that that you only want to suspend the thread when the condition is true.
• Create a condition with side-effect

Example

In this example we’ll show some simple code that simply iterate through all numbers from 0 through 99 and summarize them all. At the line where we accumulate the sequence, we’d like to print out the number and as quickly as possible continue the execution of the program.

Here is some the sample code:

   1:  package com.scispike.demo;

   2:   

   3:  /**

   4:   * Dummy class created to 

   5:   * demonstrate some debug tricks

   6:   * 

   7:   * @author pgraff

   8:   *

   9:   */

  10:  public class DebugDemo {

  11:   

  12:      public static void main(String[] args) {

  13:          int j = 0;

  14:          for (int i = 0; i < 100; i++)

  15:          {

  16:              j += i;

  17:          }

  18:          System.out.println(j);

  19:   

  20:      }

  21:   

  22:  }

The example is rather silly, but it will do for this purpose.

Steps to make the print work…

1. Create a breakpoint on line 16 ( “j += i;”)
2. Go to the debug view in Eclipse
3. Right-click on the breakpoint and select properties
   
4. Enter the code to print out ‘i’ and return false (as shown below):
   
5. Run the program in debug mode

Notice that the application now print out the i on all iterations!

Anecdote

I was quite exciting when I saw this. If you are familiar with Aspect Oriented Programming, this is quick way to patch in an aspect! Here is another silly example.

We are now patching the value of ‘j’…. I just can’t wait until someone leaves their computer on while they debug a problem…. Can you imagine the look on their face as I make some random assignments :)

Summary

In this article I shown an advanced debugging technique not commonly known in Eclipse. The technique can be used to quickly print or patch values without slowing down a program significantly (or have a lot of fun with your colleagues).

On Performance

Description

Over the last 25 years or so, I’ve been often tasked to help optimize the performance of various solutions. I keep careful records of all the work I do and I recently decided it would be a good idea to collect all my thoughts/experiences around performance.

I’m actually not only collecting notes on performance, rather I’m trying to categorize all my notes. Unfortunately, most of my notes are in old notebooks and they are filling up several shelves at my house. I’d like to get some of the  notes synchronized and made electronic so that I can make use of them in my consulting/teaching. I expect this will take many years…

One idea that I had was to organize the material around architectural viewpoints based on views that I’ve developed over the years (primarily using the IEEE 1471-2000 framework. For some of the viewpoints I literally have 10’s of thousands of notes, so I thought I’d tackle some of the smaller viewpoints first. So… for my next x blog entries, I’ll extract notes on performance.

To make these notes useful to others, it would probably be best if I discussed some of the topics that I take for granted. These are issues that my notes will not discuss because they are principles I consider them given (or immutable if you’d like).

What influences performance?

In my experience, there are only a few things that are worth focusing on:

• Algorithms and data structures. E.g.,
• Simple algorithmic improvements:
• Bubble-sort versus quick-sort
• Graph-search algorithms
• SQL query optimization
• Indexes
• Proper use of the database expression power (SQL)
• Locality of reference
• Caching
• Memory usage
• Taking advantage of immutable knowledge
• Processing power
• Scaling by adding hardware or reassigning resources

Process of performance engineering

Make sure you understand what are desired performance characteristics of the system you are trying to optimize. Sometimes the total through-put is most important, sometimes its reactiveness is most important, sometimes various scenarios have very different performance requirements. In other words, make sure you understand the problem before you start optimizing.

One of the most common errors I’ve seen is the premature and blind performance optimization. In other words don’t optimize before you know where you have a problem. Some performance optimization techniques come at a high expense from other architectural perspective (such as reuse, maintainability, continuity, …).

By premature optimization I mean performance optimization that are suggested before we even know we have a problem. Blind statements made by someone in the organization that are blindly applied across the organization. E.g., using getters provide a function overhead and hence should never be used. Let’s make the data members accessible to the client. Such a decision (apart from being wrong in many environments) has a high cost and are often not very effective.

By blind performance optimization I mean: DON’T optimize without profiling the application. More often than not, the ones optimizing are optimizing the wrong thing (e.g, who cares if you can save 1 millisecond by reducing a method call if the problem is that you are using several seconds in some database query…). 

Let me illustrate the last point with an example (an old note from my notebook). Many years ago I was working on a large scale distributed real-time system. The system we were working on had an extensive framework on which we build a large set of specialized applications for various clients. The system had serious performance problems. When an operator pressed a navigation button to move to another view, it took seconds to move to the next screen.

The framework team were called in to an all-hands to improve the performance. The performance improvement went on for many months and the engineers were optimizing based on a set of principles set forth by one of the most experienced engineers in the company. Even so, the progress was slow. After about 9 months, they managed to optimize the performance with ~ 20%. Some of the performance optimizations degraded the architectural structure (often you can steel CPU cycles by collapsing architectural layers).

My team was eventually also called in (we were working on applications, but we had a good reputation and the framework team was getting desperate). Debugging the application was rather cumbersome (at that time we were using an in-circuit emulator (ICE) that very few engineers in the organization mastered. On my team, one of the engineers had been working on the design of the ICE and knew it inside and out. Using the ICE and a profiling tool, he and I sat down and started to narrow down the areas where we spent time. This was probably the first time this problem had been attacked this way. To make a long story short, after a few hours, we had optimized the performance by 40 times. A day later, the system ran 250 times faster. After a week the system ran many thousand times faster.

How did we do that? It turned out the problem was in a few lines of code. It was an algorithmic problem (unnecessary multiplication in a loop in the display driver). There was nothing we did that the other engineers would not be able to see, nothing particularly clever, no magic. It’s more like finding your car in a large parking garage… but we ‘cheated’ by turning the lights on.

Summary

This is the introductory article where I’ll be focusing on performance optimization. The idea is to collect some of my notes from my work since leaving school. This first article was written to establish some basic principles that I will assume known in the articles to follow. Some of these principles are:

• Don’t optimize without understanding the performance goals
• Don’t optimize without a profiler (or some way of measuring performance)
• Performance comes foremost from algorithms, locality of reference and computing power

Why DSL’s are such a hard sell

Overview

Over the last years I’ve built a dozen or so large scale Domain Specific Languages (DSL) and almost without exceptions, they’ve been hard sells. Usually, each of these received initial resistance in the organizations they’ve been introduced. This despite their convincing potential and their eventual success.

Here are some examples of these successes:

• Project for Company X. Created a set of DSL’s for various aspects of their development. Two main effects:
• Average time to develop an application went from 4 man-years to 3 man-weeks (factor of 70 improvement)
• Project for State Agency Y. Create a DSL to handle a complex connection between implementation layers
• Average time to develop a connection went from ~one week to 10 minutes (factor of 240 improvement)
• Project for Company Z. Created a DSL to generate variations of application that track various initiatives (metrics not in yet, but we expect a similar improvement in development effort as in the other projects on this page)

In this blog post I’ll focus on the kind of resistance one may expect and why.

Building our own language, You’re kidding right?

The most common objection is to the presumed effort required to create a DSL. Few have experience writing a language and those that do typically wrote languages some time ago.

Fact is, if we suggested that one build a language to solve their problem a decade ago, it would have been a very ambitious project. We would have looked at a massive project taking months if not years to complete.

With today’s tools though, writing a DSL is what I usually categorize as a “weekend project”. Although there have been languages that have taken me months to complete, the typical effort is in days (or worst case weeks).

The latest tools in the open source and also in the commercial world are very different from the ones available say a decade ago. Here is a typical estimate for a language…

• Design of the language (the abstract syntax) – Typically 1-3 days
• Implementation of the concrete syntax – Typically 1 day
• Including editors with syntax highlighting, auto completion, …
• Transformers – 1 day to weeks…
• This is often the tedious part where you generate code or other artifacts from the abstract syntax

OK… the times above are not true for everyone. It requires that you have intimate knowledge of the language generation tools, but achievable I believe for any reasonably skilled engineer.

My way around the concern of effort is BUILD THE TOOL FIRST, THEN SUGGEST IT.  Because the effort is rather moderate, I can usually build a first cut of a tool over a long weekend or while flying around (no kidding, I do my best work on airplanes :)). The realization that a language can be created over such a short time and already be useful typically removes the concern of complexity.

The approach you suggest sounds a lot like a 4gl. We tried that before and it was a disaster

A 4GL and a DSL are different things. A 4GL was a general purpose language at some high level of abstraction (I have nothing against that by the way). A DSL is a special purpose language preferably at the HIGHEST level of abstraction possible. Believe me, they are different.

Again the remedy is education, but the association between experiences on 4GL’s and expectations of a DSL is real and something you must prepare for.

That is going to be a disaster for recruiting… Who would want to work on that?

Ouch… so true… and so hard to argue against… I’ve tried a myriad of ways to work around this. Let me give you a few examples.

The Parallel Approach

The parallel approach means that we create a DSL that generate code, but also allow for the developers to do the same by hand.

This has the following effect:

• We are still using the general purpose language… It is just that you can use this tool to makes you more productive.
• Eventually, the DSL becomes the dominant or the only approach

The Hook Approach

Leave something to be done in the general purpose language. Don’t hide it all. This usually make a lot of sense anyway. There is often some part of the the problem that may be best solved by a general purpose language. Typically, this is some procedural part of the problem. As long as SOME PART is written in the ‘recruiting language’ the solution may be a lot more palpable.

I’m hoping that some of these objections will disappear with time, but right now they are real and I have to admit that I’ve modified DSL’s ensure that programmers get to program in the general purpose language, EVEN THOUGH I KNOW HOW TO GENERATE THAT CODE TOO and it would have been more productive.

We make money by the hour… Why?

I’m sure some of you think I’m kidding, but believe me! This is a common response from some development domains  (typically government or contractors for government projects). This is the one argument that will usually make me give up head for the door. Personally, I don’t understand how organizations can live by those goals and openly talk about them.

Government! Change your metrics!!!

Summary

When you suggest the use of DSL'’s in software organizations, expect resistance. Very often this resistance is based on unjustifiable concerns and often the way around it is build it first… This eliminates the fear of the complexity of building a language, it also gives the stakeholders a chance to see what it would look like.

My favorite tool to show off the ideas are slightly different based on what kind of concrete syntax I want. If I want a graphical syntax, I prefer to use the Microsoft DSL tool (although, the graphical modeling framework of Eclipse (GMF) also works well). If I’m creating a textual syntax, my favorite tool right now is XText (Eclipse based tool).

iPad2 vs. Xoom

I've been eyeing the new iPad, but when I heard about the delay (and I have to admit also my current trip to Asia and the thought of not having something to play with on the plane), I decided to try out the Motorola Xoom instead.
I've been using the old iPad and briefly tested out iPad2. I've never been all that imressed by Motorola's design, so my expectation was not very high. However, after 3 days of use, I'm sold. In this blog I'll give you some reasons why.
Reason 1: It's Android!
It is so nice again to be on a platform that is not controlled by 1 company (person). I can now write my own apps without having to get it approved by the almighty fruit.
Reason 2: Flash!
Need I say more? I heard prior to buying the device that flash didn't work, but there must have been an upgrade between when I read the article and my purchase. Flash is working perfectly. This allows me to see the learning modules we have without opening my Laptop!
Reason 3: Multitasking
True multitasking... Really cool.
Reason 3: Apps
When I read previous comparions of iPad and Xoom, I saw that one of the issues people have with Xoom is the lack of apps. So far I don't see why. The selection of apps are great. All the apps (-1, a big one, Netflix) works as good or better on the Xoom!
Reason 4: The screen
It just seems sharper than the iPad..

Published with Blogger-droid v1.6.8

Retrospectiva

This post is just a quick update on the status on alternatives to the PivotalTracker.
I've been playing a bit with Retrospectiva. It seems to be a good alternative to PivotalTracker.
At SciSpike, we'll setup a free service for our customers to move their projects to Retrospectiva. The projects that we'll still involved with, we'll move to Retrospectiva before the free-period ends.
My plan is to create a converter that can take the data from existing PivotalTracker projects and move them to Retrospectiva with minimal loss.
I know that some of you probably want to continue to use PivotalTracker and I applaud that decision. I've always liked PivotalTracker and still think it is a great tool (although, overpriced)!
When I get some more experience using Retrospectiva for some real projects, I'll give you all an update of how they compare and maybe also some recommended agile practices in Retrospectiva.

PivotalTracker is going Commercial

Ouch... I just learned that PivotalTracker will start charging for their online project tracking tool.

I have mixed feelings. On one hand, I recognize their right to make money on what I consider to be an excellent tool, but on the other hand, I feel tricked...

I have a bunch of projects that I'm tracking in PivotalTracker and been pushing it where ever I've been consulting. The pricing model is a complete surprise also. a) it is too expensive, b) it doesn't fit the way I've been using PivotalTracker.

Because I've been consulting in many places, I have many collaborators and the price that I would have to pay may be as much as $3600 per year! That's a cost that I had not planned for.

I do apologize to my customers that may have started to use PivotalTracker under the wrong assumptions. I will make an investigation of alternative tools and post it on my blog as soon as I get some time to do the investigation (probably mid February as my schedule is ridiculous at this time).

Anyway, I'll be back with another post with alternatives as soon as I get around to it. Maybe I'll get some people together and build an open source alternative? Please let me know if you're interested.

Function Objects in in Java

This blog entry is really just a support entry for a YouTube movie I created today.

A few weeks ago, I taught a course on plain old Java… The students had LISP background and raised the questions:

Does Java have Closures?

Well, it doesn't really have direct support for closures directly, but we can achieve most of the design advantages that closures provide. In this short demo, I show how one may implement closures in Java.

Code Listing

Here is the final code listing for what I showed in the demo.

ClosureDemo.java

   1:  package com.scispike.demo;

   2:   

   3:  public class ClosureDemo {

   4:      

   5:      public static void main(String[] args) {

   6:          System.out.println("Gauss should have said     : " + sum(1, 100, new EchoFunction()));

   7:          System.out.println("Sum of squares from 1 to 10: " + sum(1, 10, new SquareFunction()));

   8:      }

   9:      public static int sum(int min, int max, Function f) {

  10:          int sum = 0;

  11:          for( int i = min; i <= max; i++)

  12:              sum += f.apply(i);

  13:          return sum;

  14:      }

  15:  }

Function.java

   1:  package com.scispike.demo;

   2:   

   3:  public interface Function {

   4:   

   5:      int apply(int i);

   6:   

   7:  }

SquareFunction.java

   1:  package com.scispike.demo;

   2:   

   3:  public final class SquareFunction implements Function {

   4:      @Override

   5:      public int apply(int i) {

   6:          return i*i;

   7:      }

   8:  }

EchoFunction.java

   1:  package com.scispike.demo;

   2:   

   3:  public final class EchoFunction implements Function {

   4:      @Override

   5:      public int apply(int i) {

   6:          return i;

   7:      }

   8:  }