CollabSphere 2019 Slides

A short few weeks ago at CollabSphere I was fortunate to have the opportunity to present some functionality that I use everyday in the development of the XPage application that powers our entire business.

The session was titled “Real World XPages” and the presentation is now on Slideshare:

In the coming days and weeks I will be expanding on the topics in the presentation (and adding others) as well as making the code and demos available.

Beware the lotus.domino.UserID error in Domino 9.0.1 FP8/FP9 and other upgrade musings

Over the weekend we upgraded our production Domino servers to get to 9.0.1 FP9 HF139(64-bit) & HF137 (32-bit).  Well, we “tried” to upgrade the servers.

Then all hell broke loose.

Here’s what we did:

• Server1
  • Primary XPages server with NIFNSF enabled.
  • Upgrading from FP8 HF174 to FP9
  • Upgrade failed with error “Installer found the incorrect version…”.
• Server2
  • Backup XPages server.
  • Upgrading from FP6 to FP9.
  • Upgrade failed with error “Lotus Notes/Domino or a Notes/Domino related process is still running”.
  • Booted into Safe Mode and installed FP9 and HF62
  • Upgrade to FP9 HF137 failed with error “Installer found the incorrect version…”
  • Applied JVM Patch 1_8SR4FP10
• Server3
  • Emergency XPages server.
  • Upgrading from FP1 to FP7 (in the event there were issues with FP8/FP9 – this ended up being a saving grace)

After upgrading Server2 and attempting to upgrade Server1, both servers started throwing the following exception when attempting to open any Xpage with a document data source:

javax.servlet.ServletException: java.lang.NoClassDefFoundError: lotus.domino.UserID

The only reference I could find to this specific issue was from this question on server fault.

In addition, Server1 started crashing any time we opened databases in the Notes client and started throwing long held lock dumps … over and over and over again:

Not sure if this was being caused by NIFNSF being enabled on the server?  Watching the view indexes on disk, the server was leaving 100s and then 1000s of temp .ndx files on disk as the server crashed over and over again.

Luckily, we intentionally upgraded Server3 to only FP7 in the event that we had issues with the other servers.  This saved our ass as we were able to point our users to this server with minimal interruption as we dealt with the issues above.

We were finally able to get Server1 & Server2 upgraded to FP9 HF60.

Here is what we learned in the process:

• If you are receiving the exception “javax.servlet.ServletException: java.lang.NoClassDefFoundError: lotus.domino.UserID” when opening an XPage with a document data source, then the JVM is probably hosed.  Once we were able to successfully upgrade to FP9 HF60 then everything was fine, but it doesn’t explain why this issue started on a server where the upgrade process failed initially.
• If, when trying to install a fixpack/hotfix you get the message “Error installing Domino Fixpack: Installer found the incorrect version. . .” then you probably need to install the base version of Domino, in our case 9.0.1 and then the FP you are trying to get to plus the HF you are trying to get to.  This is detailed in this technote.
• If, when trying to install a fixpack/hotfix you get the message “Lotus Notes/Domino or a Notes/Domino related process is still running…” then you may need to disable the “Windows Management Instrumentation Service” in the Services list of the server.  We PAUSED the service and successfully upgraded the server.  See this technote.

Questions from the DataTables Webinar

Last Tuesday I presented a Webinar on using DataTables with XPages which was hosted by TLCC and Teamstudio.  There was a Q&A at the end of the session, but unfortunately due to time constraints we did not get to all of the questions and kind of rushed through the questions we did get to.  So I wanted to dedicate a blog post to answering all of the questions that were submitted.  Here we go!

Q1: (Submitted by Jena Chinmaya) – Can we use Data table of this format on a regular non xpage classic notes web database ?

A1: Absolutely!  A lot of the concepts that were discussed would still apply in a classic Domino Web scenario.  The big question that would have to be answered is, how is your data being delivered from the back end to the front end?  If you are using a more recent version of Domino (say, 8.5.3 or greater) then you can create an XPage with a REST service control.  If you are using an older version of Domino, then you still have a few different options:

• Using a $$ViewTemplateDefault form to display the view.  This would be similar to using a xp:repeat in XPages since you would be applying DataTables to an existing html table of data.  I created a quick demo so you can get an idea how this would function.
• Use an agent to deliver view data via REST
• Use ajax to get data using the ?readviewentries&outputformat=JSON option of a view url.  This method would require some additional work in DataTables to read the data.

Q2: (Submitted by Chris Richards) Could you, using REST services – create one custom control, which you can drop anywhere, and just pass in a cusotm property, namely a view name so that it can be drag/dropped/re-used all over an application with ease?

A2: This can definitely be accomplished and is a great way to make adding view data as simple as drag and drop.  I essentially do this now through an extremely configurable custom control.  The key to success is making sure that the columns object gets properly defined if you are using JSON data.  I hope to make my solution available in the very near future and I’m contemplating a “light” version that I can get out quicker.

Q3: (Submitted by Csaba Kiss) Do you see any performance difference using custom REST service vs standard JSON REST?

A3: While I have not done a direct performance comparison of these REST services, I think it’s reasonable to assume that the more “custom” the custom REST service is, the more it will slow down vs the standard REST service.  For example, if while processing each row you are looping through column values to perform actions, you can certainly expect a performance hit.  If the custom REST service is just traversing a view, then I would expect the performance to be similar to the standard REST service.

Q4: (Submitted by Jen Watkins) how does one format dates? The format that comes back by default from DataTables is crazy.

A4: I think the format you are referring to is the ISO-8601 format, which you can see in the screenshot below.  Actually, this format is being delivered by Domino.  You can see this at the bottom of the screenshot in the REST response.

There are a few ways to deal with this:

• When working strictly with dates, my personal preference is to just format the column as text in the view.  DataTables recognizes the value as a date and column sorting will work appropriately.
• You can use a renderer to format the date to your preferred format.  Here is a great post on the DataTables website demonstrating how to do so.  This post references the moment.js library.  I use this library as well and it makes working with dates incredibly easy, especially when doing date comparisons.

Q5: (Submitted by Jen Watkins)  To filter data before it is returned, for example to return only the data that a particular user is allowed to see, do you have to use the CustomRestService as opposed to the ViewJsonService?

A5:  The answer to this question depends on how you have security setup in your application.  If your users are authenticating and they are calling REST services that are retrieving data from databases that require authentication, then the appropriate records will be returned based on Domino security.

On the other hand, if you are no so much concerned about security, but want to only show, for example, documents that are assigned to the current user, you can sort or categorize your view on the back end and submit the “keys” attribute with the REST service and the dataset returned will be filtered.  You can actually get very creative with using categorized/sorted views to get away from the old Notes habit of having a different view for every different categorization/sort.

Q6: (Submitted by Kevin Gregg)  What if you needed to combine data from multiple views into one DataTable – what is a good method of doing that?

A6:  When answering this question during the webinar I may have misunderstood the intention of the question so I will answer it in two ways.  This question can be interpreted in 2 ways:

1. You want to add additional rows – I actually do this a lot.  We have 2 databases that store almost identical information.  In classic Notes land, we would have to show separate views and embedded views when displaying this information to the user.With DataTables it is very easy to combine data sets, as long as the configuration of the data sets match.  What I do is load the initial data, then in the initComplete callback I make an ajax call to the 2nd set of data and then use the add rows functionality of DataTables to merge the data:

initComplete: function() {
// Check for any other data sets that need to be added to the table 
$.ajax({ 
    type: 'GET', 
    url: <url to REST service>, 
    contentType: 'application/json', 
    dataType: 'json', 
    cache: false, 
    success: function(response) { 
                var vTable = $(dataTableClass).DataTable(); 
                vTable.rows.add(response); 
                console.log("init adding records"); 
                console.log("initComplete draw()") 
                vTable.draw(); 
   } 
}); 
}

2. you want to add additional columns – This is akin to having a xp:repeat control and having a column where you perform a @dblookup to another set of data based on information in the original set.   For this scenario, the best course of action is probably to use a custom REST service or an XAgent to build the data set on the back end and deliver the completed set to DataTables.

   However, you could also accomplish this in DataTables.  In your original data set you could have one or more empty cells acting as placeholders.  Then, similar as in #1 above, in your initComplete callback you can make an ajax call to another set of data and then using a shared key between the two, insert the retrieved data into the DataTable.  I will try to work up an example of this once I can find a few minutes.

Q7: (Submitted by Miguel Angel)  How I can lock a document from a datatable to prevent the save replication conflict? How I can show a online message from a Datatables, when a customer made a change at that moment?

A7:  The lack of Document locking was certainly one of the early complaints of XPages.  Ultimately, people worked around it or rolled their own document locking mechanism.  Some of the solutions out there include:

• An OpenNTF project from Frederick Nordling
• This post from StackOverflow
• In our environment, we created our own document locking solution.  We needed to do this because we had users that were updating documents in the Notes Client as well as our XPages application.

To relate this to DataTables, if you want to initialize a lock from DataTables, you can do this by adding an onclick or ondblclick event that submits information, such as docid,  to a REST service.  To release the lock, you would do the same.

To answer your 2nd question, I’m assuming you want to update the current user if data they are looking at was changed in the back end by someone else.  Here’s what I think the simplest solution is:

• Add a setInterval to the initComplete callback that retrieves the REST data (or maybe just retrieves a column with date modified value).
• Loop through each record and compare the values to the original DataTable (or compare the date modified column).
• If there is a difference, notify the user.  If you are using Bootstrap you can use the alert classes to display a message above the table, for example.

Upcoming Webinar: XPages and jQuery DataTables

Join me next Tuesday, October 11th for a webinar sponsored by TLCC and Teamstudio – “XPages and jQuery DataTables – Simplifying View creation while maximizing functionality”

In this webinar I will show you how you can use DataTables to create rich, functional views, which is especially relevant if you are modernizing or rebuilding an existing Notes application.  While the title references XPages, it is not a requirement.  DataTables is compatible with any back end that can deliver data via REST.

You can register here:

https://www.tlcc.com/admin/tlccsite.nsf/pages/xpages-webinar?opendocument

I hope to see you there!