A simple example from today of how IBM needs to do better

We moved an application from an 853 server to a 9.01 server and users reported that multiple attachments were being uploaded into the XPages attachment control.

It seems to be related to this bug in 9.01

LO82247: PARTIAL REFRESH OF A DIFFERENT PORTION OF AN XPAGE CAUSES THE CONTENT OF A FILE UPLOAD CONTROL TO DUPLICATE.

Which has been closed with no resolution to be fixed in a future release.

So I have had to

  1. Open a ticket here
  2. Identify the issue
  3. Remove partial refreshes which has a negative impact on uasability
  4. Test the “fix”
  5. Submit the change to source control
  6. Deploy the fix to the server
  7. Inform the users that a workaround has been put in place while IBM fix the bug
  8. Then in the future reverse steps 3,4,5,6,7

This is not a sustainable way to work.

 

ICONUK and the State of Domino Web Development

Caution Long Post, I’m stuck on a train.

The best User Group yet.

I am just back from a very impressive ICONUK. I can honestly say that it was the best LUG that I have been to. The best part for me was the informal discussions within the community. It is a time of change and the community discussions were real, interesting and honest.

I came away with more questions than answers but my new questions are better informed and more useful than the set I arrived with.

Why I went

In all honesty I went to ICONUK hoping to understand what platform we should move to ( from the experiences of other community members rather than IBM ). I didn’t come away with an answer but I did came away with a much better understanding of my options and even a new option in the form of ASP.NET MVC ( never thought I would say that ).

XPages does a lot

More importantly I came away with an understanding of just how good Domino / XPages actually is compared to other stacks.

I know that it has some ( fairly serious ) issues but it does do a lot more for us than perhaps we give it credit for, particularly for those of us that need / want to minimise the amount of front end work that we need to do.

I think that the lack of a clear alternative is an indication of how good XPages has been, particularly with RAD ( Rapid Application Development ).

To move to a separate back end and front end requires two different skill sets and synchronisation of changes on both ends whereas with XPages one developer can do it all. Security is much easier (although authentication is often not) and the database , app server and web server are all rolled into one.

So why change ?

So…. lets go back to basics for a minute :
Why do I want / need to find a new stack ? Just to give some context I have no interest in email and run a niche SaaS provider whose solutions are all currently built on XPages.

For me there are 4 drivers :

1 . I cannot take the risk of running my business on a dead platform.

A real concern that the future of XPages is linked to the future of Notes and Notes is rapidly disappearing from the UK.As Notes disappears the IBM resources allocated to XPages are clearly diminishing – I mean the development team, the evangelists, the product managers.

The community resources are also diminishing too.Modern web application platforms need constant maintenance – its not like letting Lotus 123 linger on in the comparative safety of a walled garden until 2014.

IBM sometimes make the point that “XPages is complete” – I very much disagree, try doing source control for real.

2 . XPages is no longer fit for our purposes ( yes we are moving the goal posts ) and there is no demonstrated intent to make it so.


The XPages platform is not moving forwards.IBM have said that XPages is complete so it is only natural that there should be less people working on it ( as there are with lotus script and formula ) – lotuscript and formula have effectively been deprecated by XPages so that is OK – XPages has not been deprecated.

A modern web application server cannot be left at a point in time and called complete.Java 6 and the constraints of the NSF ( compared to Cloudant as an example ) are the biggest demonstration of this. As we develop our competencies with Java ( which many XPage developers have ) we increasingly want to use external modules and improved Java techniques. Having to use a version of Java from 2011 is a constraint and as much as I admire the OpenNTF Java API project it needs to be in the core product.

Likewise we are having to do all sorts of whacky work arounds to deliver faceted searches, activity streams and other rich data streams that other platforms can deliver much better by using modern databases.

It was great to talk to Frank van der Linden who has been working with Cloudant as part of his winning entry to the developer competition. He is full of enthusiasm for what he has discovered in Cloudant – this is great but why is he the only one ? Why wasn’t there a session at ICON from IBM about Cloudant and why is Frank having to write the boiler plate code to integrate Cloudant with Domino and lastly why isn’t Cloudant part of Domino ?

3 . Dojo 2 and passive maintenance

I have a concern that Dojo might become the thing that kills XPages. Did you know that Dojo 2 is due in Fall 2016 – here is the github progress – what plans does IBM have for Dojo 2 ?

I am not a front end developer but it is obvious that a) XPage is hugely dependent on Dojo and b) Dojo is much less loved than it was – what happens when Dojo stops working ? This could be a totally bogus notion so please correct me if I am wrong.

4. Technical Debt

This is about the amount of time that lapses between now and when we move platform.

Every new feature that my team adds to our products creates new technical debt because we will have to re-develop those features on a new platform – we operate a SaaS model and this debt has to be addressed at our cost.

The reality of this is it has taken us 5 iterations of XPage applications to settle on a really good approach and it is inevitable that our first few attempts on a new platform will also need to be re-written several times as we become experts on that platform.

So….. where does this leave me ( and maybe you ).

The first thing to say is that our ( ICS ) situation is in some ways no more severe than those who will need to move from Angular 1 to Angular 2 or those whose wordpress sites no longer run after PHP 5 ( I have 2 such sites ). The pace of change in web application development means that step changes happen and platforms get deprecated – in a way we were spoilt with an IBM platform that could run a version 3 application on a version 9 server with no problems.

Having said that people on Angular 1 can see that there is Angular 2 and this takes us to the crux of my problem.

The crux

The IBM people that I spoke to over the last few days genuinely want to believe that there is a significant announcement going to be made about “Application Development” in Q4. They are asking for our ( the communities ) patience and speak genuinely of understanding that the current situation is unacceptable.

They also speak in a way that acknowledges that the App Dev side of Domino is something in itself with it’s own user base independent of mail accounts. In informal discussions they speak of good reasons why the XPages resources have not been available and how this was a business necessity to deliver other much needed work ( not Toscanna although I have heard rumours otherwise ).

What is clear is that none of the IBMers are “in the know” and that is where I get worried. How can IBM pull a rabbit out of the hat if there has been limited consultation with users or their own business team ? In conversation everyone just defers to Ed Brill as the person who will decide what needs to be delivered.

Now to be fair to IBM none of us saw XPages coming or contributed to that concept ( which in itself came out of Workplace ) and yet it was absolutely the saviour of Notes development in 2011. This is the article I wrote at the time about XWorks ( a very positive thing for me ) but it is interesting to reflect on where we are now as a result of XPages => IBM’s XPages App Server Dilemma – a faster horse, a cash cow or a compelling new concept ?

On the flip side the experiment that was XPages on Bluemix was in my view a miss step because it has soaked up valuable resources and community energy and delivered nothing that is useful today. To be fair to IBM they insist  that it was necessary to show that XPages was still being taken seriously, I disagree but I can see some logic.

What would it take to win me back ?

IBM are asking for our patience while they develop a plan for App Dev.

They claim that they have changed and that our trust will be well placed. What would it take for me to stay with XPages ……… ?

It would take a significant step change in XPages capability in a timely manner and with IBM evangalist support to help the community to exploit the new capabilities ( and stop the haemorrhage of users )

I would want need to see :

  1. A Clouldant / CouchDB style database in the XPages Core product ( on premises ) that leverages the existing Java APIs ( @dblookup etc ) and has new ones so that we could develop performant applications with larger data sets, faceted filtering and dynamic charts ( instead of cached ones ).
  2. Java 8
  3. Source Control fixed ( elimination of time stamps etc ) in the core product – not a hook for an OpenNTF project ( as good as that may be ) – if nothing else to demonstrate that IBM does have ownership of its unfinished sub standard issues.
  4. A technical evangelist who can help us exploit this new functionality.
  5. All of the above by the end of Q3 2017 with demonstrable proof of progress in the mean time so that IBM can earn the trust of their customers.

The Fix Pack V’s Version thing and 2021

I really can’t get excited as to whether it is fix packs or versions as long as it is done well. There are clearly a lot of things that have not been thought through ( what do we call interim fixes ? how is certification handled ? ) but… if it were a robust continuous release process I would have no issues.

IBM claim that is is saving many many hours so lets see them put those man hours to good use. As for 2021 5 years is a long time in IT and in any case if the product stops evolving it is a moot point anyway because it won’t stay fit for purpose as the world moves on.

In Summary

Thanks to Tim Clark and his team – typically they don’t publish a list anywhere – they just deliver :-), the sponsors and the speakers and IBM for hosting the event.

Domino does a hell of a lot. If a new startup took it as a concept to a Silicon Valley venture capitalist I think that they would be impressed. Replacing it is turning out to be more difficult than we thought but as the need to replace it grows more people will rise to the challenge.

IBM is asking for a chance to make things right and say that they understand that it needs to be a significant step change. I have described above what it would take to convince me and am willing to put my scepticism aside and stay positive- but it is a “one shot chance” as an IBMer said today.

What would it take for IBM to convince you ?

The moral of the story – do some monitoring

We have been battling with one of 6 Domino XPage servers that is running a consistently higher load average than the other servers. The performance was OK from a customer point of view but differences in systems are always a worry.

2016-08-16_08-57-35

We tried everything I could think of – probably spending 2 – 3 days on this overall.  We checked the XPage application, we checked Domino configuration, we even moved it to a different host server and we even rebuilt it from a (new) standard image. Anyhow, the problem has been solved 🙂

We recently did a routine update to the linux OS and the issue went away. It seems that there was a bug in the linux release that we were using which –

“Due to prematurely decremented calc_load_task, the calculated load
average was off by up to the number of CPUs in the machine. As a
consequence, job scheduling worked improperly causing a drop in the system
performance – https://rhn.redhat.com/errata/RHSA-2016-0494.html

So the morale of this story

  1. Do some monitoring, that way you can see how updates to the OS and other software are affecting performance. If you roll out an update look for improved or decreased performance – always being on the latest release is not necessarily the solution.
  2. Never build a standard recovery image and assume that it will run correctly in operation unless you have run it in production. This server was first based on our latest build template ( including a new OS version – which was broken ) and when we rebuilt it we used that image again ( still with the broken OS )

We are using the Opsview Atom monitoring suite which is based on the Nagios open source platform. It is fairy good ( and well priced ). I will be post some articles on it soon.

loadaverage

Auditing for IBM Notes mail rules sending emails externally

A customer asked us to help with a tool to audit for mail rules that were forwarding emails externally. I had a look around and found the very good openNTF Mail Rules Analyser application from Thomas Hampel

Having looked at it David Harding and I felt that it was great starting point but we wanted something a bit different that could be used by an auditor without admin access and which showed the history of mail rules.

We modified the application and are hoping to feed back some or all of the changes into the openNTF project. In the mean time you can down load it from this link Mail Rules .

2016-03-11_12-02-19

Imagine Wikipedia for your own Assets – Asset Wiki

Think of Wikipedia but with a secure page for each pump, centrifuge, vessel etc. that you operate.

Now imagine all of the knowledge that you have in your organisation about these assets consolidated onto a single page per item – Root Cause Reports, Maintenance Strategies, Maintenance Plans, Spares Reviews, Criticality Rankings, Build Plans, Inspection Reports, Actions, Historical Costs, Projected Costs, emails etc.

This is Asset Wiki™ – a way of consolidating your asset knowledge into one place so that you and your teams can make better, more informed decisions, more efficiently. Continue reading “Imagine Wikipedia for your own Assets – Asset Wiki”