Salient Soft

Various points/gotchas worthy of note, all discovered whilst learning about React…

1/ You can define components in their own classes – to do this you extend the component class and define the render method. However, this is now considered legacy in favour of just writing a function which returns the necessary jsx/tsx. This is discussed here, and the migration of components from legacy classes to using functions instead is discussed here.

Comments Off on React – Framework Points to Note

Initial points found upon trying out React…

1. Deploying and running a Basic React App

The React framework site and nearly all the other tutorial sites I found do not make it clear at all how to deploy/run one of the react examples to a static web server, much like Angular does by default, with the front end framework bootstrapped from index.html. In most cases, they suggest using Node.js on the server to run the app, which then creates everything dynamically and just lets you access the site e.g. via localhost:3000 if running locally.

Whilst this may be efficient if you have node.js on your server, it does not help when deploying to a static web server as I would be e.g. with Zen internet, or when extending an existing web app with some react on one of the pages.

After quite a bit of head scratching, I found this simple post which showed how to simply add the “homepage”: “.” directive to package.json, which defines the homepage for the app. This is picked up by the webpack build, and an index.html is created (or extended) with the necessary javascript to start react. I tested this out by amending this simple react calculator tutorial app.  It built correctly by adding the necessary webpack bootstrap code to the public index.html template in the app, and it ran fine both locally with http-server and on zen internet. It also ran correctly locally via npm start, which runs it dynamically under node where this is available on the server.

In my case I had deprecated OpenSSL build issues as detailed on this post here. I worked around these purely just to get the example to build and run as it was my very first stab at running react, by using set NODE_OPTIONS=--openssl-legacy-provider as per the post. Note that this is a dangerous legacy setting which should not normally be used as it opens up legacy SSL vulnerabilities. For my simple initial test this was not a concern however.

1. Creating and building a React App from scratch

I used create-react-app for this which looked like a really simple tool to create a standard build simply, somewhat like the angular CLI. This post here shows how to do this with typescript enabled by default, by simply using the following npx command:

npx create-react-app my-app --template typescript

This created a simple app which again I could modify to be served from a static server as above via the homepage directive. However, On researching this further, I noted that create-react-app is now deprecated.

I am continuing to use this for basic react learning, but going further forward, I will likely be adopting Next.js as the framework to use with react and to build apps with. The react site says that one of the recommended additional frameworks should now be used and Next.js seemed best for me initially, as it works with static sites and as a wikipedia presence. Remix among others is popular but did not have a wikipedia page, and I liked the static site emphasis of Next.js. However as I wanted to learn one step at a time and not hit 2 frameworks in one go, I intend to continue with create-react-app for the basics at first.

Comments Off on React Framework

My PWAs dynamically create and load a url cache in the service worker, when the installation event fires.
As this cache contains all the files in the dist/project-name directory tree (i.e. the entire deployment) for an angular app, I was keen to create the script containing the list of files to be cached dynamically as a post build step, so that the post build script could be generic and adaptive to the build contents.

This turned out to be pretty straightforward. The following points are noteworthy:

1. I wanted to dynamically create a JS file which defined an array containing the urls of all the files to be cached. This script would then be included by the service worker to be used when it created and loaded the cache, during its install event.
2. The above JS file was imported by the service worker script. I had some issues getting this to work via modules/module support, and noted from this post here that using the importScripts function instead (e.g. importScripts(‘pwa-cache-urls.js’);) gave better cross browser support/backwards compatibility. I therefore switched to using importScripts and it all worked fine.
3. glob.sync was used to generate an array containing all the deployed files to be cached, excluding directories using {nodir: true}. This was then mapped/formatted as required into a string containing all the script for the file.
4. fs.writeFileSync was used to create the script file from the above string.
5. The project name and the generated script file name were passed as arguments to the post build file, in package.json. Whilst I did look for a dynamic way to derive/lookup the project name, an easy way to do this was not obvious and there was no loss of flexibility in just passing it in the postbuild script definition in package.json, along with the name of the dynamically created script file.

This all worked fine, and I was able to create a generic post build script that would be able to discover the files to be cached dynamically and create the script for them, for any angular project I was working with.

The prototype example code for the post build is below:

const glob= require(‘glob’);
const fs = require(‘fs’);

const args = process.argv;
validateArgs(args);
const projectName = args[2];
const cacheJSFileName = args[3];

const pathPrefix = ‘dist/’ + projectName + “/”;
const cacheJSFilePath = pathPrefix + cacheJSFileName;

fs.writeFileSync(cacheJSFilePath, buildCacheJSFile());

function validateArgs(args) {
  const argErrors = [
    “Project Name and Cache Filename were not passed to post-build.js”,
    “Cache Filename was not passed to post-build.js”
  ]
  if (args.length < 4) {
  throw new Error(argErrors[args.length – 2]);
  }
}

function buildCacheJSFile() {
  const mapPathDelimiters = (path) => path.replaceAll(“\\”, “/”);
  const mapPathPrefix = (path) => path.replace(pathPrefix, “”);

  const cacheJSFilePrefix = “const CACHED_URLS = [\r\n ‘./'”;
  const cacheJSFileSuffix = “\r\n]\r\n”;
  const formatCacheJSFileEntry = (file) => “,\r\n ‘” + file + “‘”;

  const cacheFiles = glob.sync(pathPrefix + “**”, {nodir: true});
  let mappedCacheFiles = cacheFiles.map((file) => mapPathPrefix(mapPathDelimiters(file)));
  let cacheJSFile = cacheJSFilePrefix;
  mappedCacheFiles.forEach( file => cacheJSFile += formatCacheJSFileEntry(file));
  cacheJSFile += cacheJSFileSuffix;
  return cacheJSFile;
}

Comments Off on Progressive Web Apps – Dynamic Cache Script Creation

Tutorials/examples used

1. Simple tutorial here on free code camp
2. Building Progressive Web Apps by Tal Ater. This includes a source example on github.

Issues/Points Found:

1. My angular code has a number of cases where a query string is passed to a url. When caching a url or searching for one in the cache, it is important to remove all query strings and “#” fragment identifier. All my urls which are added to the cache are constants from a js/json file, and so no explicit removal is needed in the code. For searching, I use the {ignoreSearch: true} option on caches.match which strips all these on searching. Initially I appeared to have issues getting this working on all browsers, especially mobile, but in due course I established that there were other bugs at play, and in fact the above option works reliably across all browsers, including chrome and firefox on android.
2. A valid manifest .json must be declared in index.html, and this must have at least one icon of at least 144×144 in size. This is a PWA rule.
3. A promise was not being returned to the install event in free code camp example – this was due to braces being used in the install event code, but no explicit return was used to return the promise. The code did use a waitUntil in the install event, so clearly was expecting to wait for the cache additions to complete. The event waitUntil will therefore get an undefined return and not wait for the promise (i.e. the addAll to the cache) to complete. The PWAbook example above does do this correctly, and the code is very similar, but I copied and pasted code from each of the above in various examples and did not notice the missing return keyword in the free code camp example. This was subsequently remedied and all my code.
4. Service worker are always installed immediately after registration see here:
   I have not always seen this. I have often failed to see the console logs for the install event and the cache add/creation on the web site devtools console, but I suspect this is due to an existing service worker being used rather than a new one. The service worker has its own dev tools page and having just loaded a PWA web site locally on a new unused port, I noticed that the install event console logging was visible both on the dev tools for the pwa site and for the newly created service worker. All of this happened before any attempt was made to install the PWA app via the browser.
   Service workers are reused unless terminated/removed, even when the web app is updated. However you can force a new service worker on update by visiting the Application/service workers menu option on the Application page of the dev tools, and ticking update on reload. I did not prove this/try out in detail, but I did leave it on normally when developing.
   This issue may also be due to timing issues due to the bug in 1/, the promise not being returned therefore the code does not wait for all the urls to be added to the cache.
5. Service workers may be viewed via the following chrome urls:
   chrome://inspect/#service-workers
   chrome://serviceworker-internals/
6. For android, there are no dev tools available, but you can enable usb debugging. Visit the developer options under settings, and then enable usb debugging. You can then plug the phone into a windows pc via usb, and visit chrome://inspect/#devices to list the available usb devices, and connect to them, which opens a dev tools window.
7. Note that it takes up to a minute or so after the addAll is performed to the cache for it to appear as populated in the Applications tab of the chrome inspector. This was misleading at first as it looked as if it had failed/not been performed when it actually had.
8. Initially I did often see errors loading certain urls into the cache. To log this, I switched to using a loop to iterate the cache urls array, and used sequential cache.add() calls rather than cache.addAll(). This allowed logging explicitly of the ones in error (cache.addAll did not give detailed errors about the urls that had failed). Cache.addAll() is better though as it performs the adds in parallel and so is faster, so having got the pwa working I have reverted to using cache.addAll. I suspect these errors were due to code issues, in particular the above timing issues due to the missing return keyword.

Having battled through all these issues, I was able to get reliable operation, both online and offline, and both on windows and Android, with both Chrome and Firefox.

Comments Off on Progressive Web Apps – Technical Points/Issues Found

I just hit this when editing a couple of existing posts.

All the post content except the header had disappeared, and I did not seem to be able to recover it. Whilst trying, I did receive an error once about the block format, indicating there was a bug with the block layout additions in WordPress. It is possible that there is also an issue with an existing plugin/plugin version that I am using.

In the end, I discovered a reasonable workaround. When editing the post to update it, click the triple dot menu in the top RHS of the right hand pane. Then select the code editor rather than the visual editor. This revealed all the existing content but in raw code mode.

When I then clicked on the Visual editor again, all of the content came back as originally posted.

I looked for solutions/workarounds online but could not find anything concrete. Whilst this is a nuisance, I can live with the workaround for now, until I can find the time to give more effort to a proper fix/solution.

Comments Off on WordPress post content has disappeared when trying to edit existing post

I started looking into PWAs as an alternative to the Ionic framework which I have used previously to prototype an offline/online mobile application for my mobile places guide.

My main issue with frameworks such as Ionic was the need for multiple application codebases – one admin application for web based use online (which could also be used on mobile when online), and another application for offline or online use on mobile when actually out in the sticks and looking for a local place that best meets my already specified favourite criteria.

The immediate attraction of a PWA was the simplification due to a single codebase framework. As it is browser based, it can run with any web framework/app architecture and be usable online, and installable for offline use. Some of the main benefits are as follows:

• Allows a single application framework such as Angular or React to be used for a web application which can be used online or installed for offline/online use just like a native app, i.e. can be run as an app from the home screen on a mobile. This allows a single application framework/codebase to be used for all scenarios. This was the key advantage for me – the need for multiple codebases/frameworks when developing an app as a small enterprise is a serious limitation. There is also the potential need for multiple offline applications/frameworks to support all the required platforms, so there can even be a need for maintaining multiple offline native app codebases.
• PWAs are widely supported across browsers/platforms and are a standard.
• A PWA greatly simplifies the installation process. One major issue with web sites which have a mobile app is the need to visit an app store and go through the install process before being able to continue. This really breaks the flow of use.
• Web sites are really keen for users to install their app, as this offers functionality such as offline working and push notifications even when the app is not active/not in use. However, they typically resort to an pattern known as a full page interstitial ad. This interrupts the user, often with a full page overlay, right when they are interacting with content on the web site, in an attempt to force them to install the app – commonly called the door slam antipattern. This takes a number of steps, followed by actually running the app and returning to their previous context. A survey by Google showed that a significant percentage of users just dump the app installation and their web site activity completely when presented with the door slam.
• PWAs have a much more seamless install process which is performed with a click directly within the browser. It does not involve visiting an app store, installing an app, and selecting options/enabling app permissions. Therefore significantly less users are put off by this.

Initial tryout/Proof of Concept

I branched the codebase for my existing Property Details application. This was a good example to try, as it has significant complexity:

• Web application using the Angular framework (v13 at the current time).
• PrimeNG component framework for angular is extensively used, including complex components such as the photo gallery
• Dynamic scripting/script switching for configuration and PrimeNG theme switching.
• Use of the angular APP_INITIALIZER to initialise/configure the app prior to angular bootstrapping, and use of angular/browser polyfills.
• Use of the Webpack module bundler to bundle and install the application

I felt that if I could get this application to run seamlessly as a PWA with minimal effort and changes to the existing code, this would be a valuable proof of concept to justify taking PWAs forward.

My conclusion was very positive – whilst I had a number of technical/learning issues to overcome, I succeeded in getting the property details app to run as a PWA both online and fully offline in Chrome, edge, and Firefox, on both Windows and Android, with minimal code changes. The only code interventions were to add the necessary support for PWAs, which were minimal. None of the existing code had to be modified. I just needed to add a json PWA manifest file, a couple of scripts, and an icon image. I then just needed to modify the index.html page to declare the manifest and call the first of the scripts. Going forward, I will need to try the other PWA features such as background/offline sync, push notifications, and service worker to page/Window to service worker communication. I will also need to make some effort to smooth the app rendering to bring up a page quickly, especially when offline, as the current POC is somewhat jerky on occasion, with some delays on rendering. However, it works reliably as expected.

I also have not looked into the database functionality re offline/online use and syncing. For the database, it is tempting to stick with CouchDB, as PouchDB is available as a wrapper for IndexedDB and could be used for background sync with a CouchDB as I originally planned with my Ionic POC. One issue which needs exploring on this is the availability of CouchDB hosting – my ISP, Zen Internet, use CPanel which does not host CouchdB. If this is an issue I would need to either use IndexedDB with e.g. MySQL at the back end and my own abstraction/syncing between the two, or use continue to use PouchDB still with a MySQL back end – this whole area needs investigation.

I have therefore decided to look into using PWAs further as a replacement for my existing Ionic offline POC/tryout.

Comments Off on Progressive Web Apps

Following on from my previous post on dynamic routing fundamentals, the ability to route dynamically across angular elements, i.e. to use the router to route to a dynamically loaded angular element, seems to be a sticking point.

Previously, I developed a web components example where a ‘fabric’ component was responsible for dynamically creating web components and displaying them. This did not use angular routing, but dynamically added them to the dom based on menu selections or url parameters. Subsequently, I wanted to investigate whether I could use angular routing to implement this instead.

I have found a number of examples of varying degrees of complexity, which go at least some of the way. However, I have not seen what I would term a reasonably straightforward, out-of-the-box solution, which does not involve too much deep diving into the internals of angular.

The following examples have been examined to date:

A complex dynamic one here with perhaps useful ideas but actual components still appear to be declared in the angular app:
https://medium.com/ngconf/dynamic-component-generation-in-lazy-loaded-routes-d2e0f9e89c57
The code is here:
https://github.com/theAlgorithmist/Angular8LazyLoadDynamic

Some specific ideas in this one here but still some way to go IMO…
https://medium.com/@timon.grassl/how-to-use-routing-in-angular-web-components-c6a76449cdb

This one perhaps looks the most promising, and may have all the relevant techniques needed, but It still appears to have statically declared component objects which are used dynamically in a dynamic order…
https://snyk.io/advisor/npm-package/angular-elements-router

Some more research is needed on this as I am keen to fine the simplest solution which uses as little as possible of the deeper features of angular.

Comments Off on Angular – Dynamic Routing between Angular Elements aka Web Components

One of my goals is to be able to dynamically create routes based on configuration data, typically loaded at pre-init time using an app initialiser.

Two approaches I have seen to doing this are as follows:

1. Using a custom Injector to use code to inject the routes.
   This is done in this example here. The code for it is available here.
2. Using Router.resetConfig() to change the routes after initially creating them.
   This is done in this example here. The code for it is available here.
   It is also done in this example here, in the app initialiser.

Both are dynamic, but method 1/ with the injector runs before any app initialiser, so would not be suitable for my needs (I verified this with a simple test with console logging). If this were used, any dynamic config would need to be fetched within the injector code or called directly from it. Method 2/ can be used after the app initialiser – I tried calling it successfully from the ngOnInit lifecycle hook in the app component which worked fine.

This example here uses method 1/, and the code for it is on github here. I downloaded the code and modified the original version, as it contains errors in the time of day checking code. I then saved the project to bitbucket. I branched it and modified it again to use resetConfig as above from ngOnInit, and removed the injector mechanism. This also worked fine.

Whilst these approaches allow dynamic creation of the routes, they assume that the routed components are known to the application or declared in it in some form. The basis of the standard approach to routing is that the component class referred to in the route is looked up, and its selector is extracted and used to create the component tag dynamically, inserting it after the appropriate <router-outlet>. This would not work in the case of angular elements, i.e. W3C web components. These are entirely dynamic and my examples which use these in the past have the element name/component tag defined dynamically as part of the config, along with the names of the scripts to be loaded to create the web component. Another approach will need to be investigated to look into routing of angular elements.

Comments Off on Angular Dynamic Routing

These points were found whilst trying some angular router examples.

Examples tried were these. Note that they are all on Stackblitx, and therefore zips can be downloaded.
It is not obvious how, but just make sure the project panel is open by clicking on the 2-page icon near the top left. Then click on the small cloud icon containing the down arrow, on the RHS of the project panel title.

1. https://www.tektutorialshub.com/angular/angular-routing-navigation/
2. https://stackblitz.com/edit/angular-ivy-spsz1g?file=src%2Fapp%2Fapp.component.ts
   Note that this one above is angular 15 so pretty up to date

1. Basic one with just 2 router states/2 buttons
   https://angular.io/guide/router-tutorial
2. Full Tour of Heroes tutorial
   https://angular.io/guide/router-tutorial-toh 
   This above one is angular 16 so latest

1. Routes are defined using RouterModule.forRoot. In some examples the route definitions/routes array is defined in app.module.ts (as in the basic angular.io example above). More typically the routes array is defined in a separate file to separate the concerns, such as app-routing.module.ts or app.routes.ts. It can then either be exported from this file and referenced in app.module.ts via RouterModule.forRoot (as in the tektutorials example above), or RouterModule.forRoot is itself also in the separate file defining the routes (as in the full Tour of Heroes example above, in app-routing.module.ts).
   
   
2. In all the examples, routing is done by creating a component and placing it as a sibling of the <router-outlet>tag, immediately after <router-outlet>. The created component is never placed as a child of the <router-outlet>. As per the full Heroes tutorial, <router-outlets> can be nested where required – a component activated from a <router-outlet> can then use other routes to activate further child components etc.
   
   
3. When refreshing e.g. a child page on zen apache or http-server, I got 404 errors. It is necessary to tell the server to redirect to e.g. index.html or whatever you are using, to get the routing to work. This was not the case when using “ng serve” as this takes care of the issue for you automatically. The same problem happens if you enter a url manually or from a link, to go to a detail child route page. This angular.io deployment page explains how to do this for various servers, including apache as used by Zen in my case. In my case I tested on Zen and needed to edit .htaccess as per the instructions. For this to work, I had to change the path in the instructions to the correct subdomain index.html file or it still failed. I also created a child .htaccess file in the subdomain folder where my app was hosted. This ran in addition to the main .htaccess, and it all worked fine. I did notice that the lowest child hero pages on the full tour of heroes example above still failed with a 404. However, they also failed with a 404 when served via “ng serve”, so I concluded that this was a coding issue in the example and did not investigate further. My .htaccess for the full heroes example under zen was as follows:-

RewriteEngine on

RewriteOptions inherit

# BEGIN Angular Routing

RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -f [OR] 
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -d 
RewriteRule ^ - [L] 

# If the requested resource doesn't exist, use index.html 
RewriteRule ^ /angular-router/angular-full-heroes-example/index.html

# END Angular Routing

When a route is activated, its target component is always recreated and any previous component for the corresponding <router-outlet> is destroyed. I confirmed this with all the above examples by logging in the ngOnInit and ngOnDestroy life cycle methods. Components are not switched in and out by showing and hiding.

Comments Off on Angular Routing

I was using Spring Data JPA with Oracle as part of an export process from CouchDB to Oracle. The CouchDB export application first exports my existing Places data from CouchDB, then maps it to the entity model I am using for JPA/Oracle, then persists it into Oracle, and finally performs some rudimentary listing of the Oracle data by logging it.

Some issues/points came to light during this excercise which I am noting here:-

1/ My connect string to Oracle was using a Pluggable database in Oracle XE. This needs to use a service name (xepdb1) rather than a SID (xe) for the connection. To do this I needed to use a slash before the service name in the application properties, to indicate a service name, rather than the colon I would normally use for a SID:

# Oracle settings
spring.datasource.url = jdbc:oracle:thin:@localhost:1521/xepdb1
spring.datasource.username = places
spring.datasource.password = xxxxxxxx

2/ I changed the Hibernate naming strategy in the application properties to prevent unwanted snake casing of Oracle column names:

# Hibernate settings
spring.jpa.hibernate.naming.physical-strategy = org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl

3/ For persisting to Oracle, I had the choice of using Spring Data JPA, calling save() or saveAll() on the Individual Repository interfaces that extended CrudRepository (such as PlacesRepository), or I could use the Entity Manager. The Spring Data methods would persist or update fine, and could also save an iterable in one call. However, I had some small additional logic as I was passing and returning a Map. As I chose not to put this logic in the service layer, I therefore used an ordinary JPA Dao (repository) class, and autowired the entity manager via the @PersistenceContext annotation. I could therefore incorporate my logic in a single call and use entityManager.merge() to persist. I felt that situations vary and that it was not always appropriate to use Spring Data Interfaces for some use cases. However the Spring Data find queries were useful and clever for querying the database as they allowed for example a find by key with a like, and order by, just by declaring the appropriate method calls in the interface. I also used the @Query annotation for a custom query, which again was just annotated on a method in the interface.

4/ I had issues whereby the foreign keys on dependant entities were not being auto populated and threw up as null, even when I set nullable=false on the @JoinColumn annotations. In the end it was a simple case of not populating the objects in my model on both sides of the join before persisting – i.e. I was populating a list in a parent object for a one-to-many, but not populating the parent in the dependant object. Doing this resolved the problem and everything persisted automatically with a single parent persist/merge, with hibernate allocating primary keys via Oracle sequences, and then auto populating them in the foreign keys. This stack overflow post here pointed me in the right direction on this. I kept the nullable=false annotations where appropriate to correctly match the relationship I was using, and what was in Oracle.

5/ Once I was at the point of listing everything I had persisted, using Spring Data queries and some simple logging, I was initially hit with Hibernate multiple bag fetch exceptions when listing the data. At this stage I had annotated my repository methods as @Transactional rather than the service layer, and was therefore using explicit eager loading annotations as I needed everything to load before the methods returned. This post here details the issue, and it appears that Hibernate can only eagerly load one collection at a time. My solution to this was to take the more normal route of annotating my service methods as @Transactional rather than the repository, which then allowed a number of repository calls to be made from a single service method, all in the same transaction. Whilst this did expose the service layer to the transaction architecture and javax.transaction package, it is the more normal approach and gives flexibility in the service layer. In my case I could then just revert to the default lazy loading and perform the logging in the same service layer method (which I was happy to do as this was a very basic example just to demonstrate fetching the imported data from Oracle). Everything then worked fine.

Comments Off on Using Spring Data JPA with Oracle