Salient Soft

Various points/gotchas worthy of note, all discovered whilst learning about React…

1/ You can define components in their own classes – to do this you extend the component class and define the render method. However, this is now considered legacy in favour of just writing a function which returns the necessary jsx/tsx. This is discussed here, and the migration of components from legacy classes to using functions instead is discussed here.

Comments Off on React – Framework Points to Note

My PWAs dynamically create and load a url cache in the service worker, when the installation event fires.
As this cache contains all the files in the dist/project-name directory tree (i.e. the entire deployment) for an angular app, I was keen to create the script containing the list of files to be cached dynamically as a post build step, so that the post build script could be generic and adaptive to the build contents.

This turned out to be pretty straightforward. The following points are noteworthy:

1. I wanted to dynamically create a JS file which defined an array containing the urls of all the files to be cached. This script would then be included by the service worker to be used when it created and loaded the cache, during its install event.
2. The above JS file was imported by the service worker script. I had some issues getting this to work via modules/module support, and noted from this post here that using the importScripts function instead (e.g. importScripts(‘pwa-cache-urls.js’);) gave better cross browser support/backwards compatibility. I therefore switched to using importScripts and it all worked fine.
3. glob.sync was used to generate an array containing all the deployed files to be cached, excluding directories using {nodir: true}. This was then mapped/formatted as required into a string containing all the script for the file.
4. fs.writeFileSync was used to create the script file from the above string.
5. The project name and the generated script file name were passed as arguments to the post build file, in package.json. Whilst I did look for a dynamic way to derive/lookup the project name, an easy way to do this was not obvious and there was no loss of flexibility in just passing it in the postbuild script definition in package.json, along with the name of the dynamically created script file.

This all worked fine, and I was able to create a generic post build script that would be able to discover the files to be cached dynamically and create the script for them, for any angular project I was working with.

The prototype example code for the post build is below:

const glob= require(‘glob’);
const fs = require(‘fs’);

const args = process.argv;
validateArgs(args);
const projectName = args[2];
const cacheJSFileName = args[3];

const pathPrefix = ‘dist/’ + projectName + “/”;
const cacheJSFilePath = pathPrefix + cacheJSFileName;

fs.writeFileSync(cacheJSFilePath, buildCacheJSFile());

function validateArgs(args) {
  const argErrors = [
    “Project Name and Cache Filename were not passed to post-build.js”,
    “Cache Filename was not passed to post-build.js”
  ]
  if (args.length < 4) {
  throw new Error(argErrors[args.length – 2]);
  }
}

function buildCacheJSFile() {
  const mapPathDelimiters = (path) => path.replaceAll(“\\”, “/”);
  const mapPathPrefix = (path) => path.replace(pathPrefix, “”);

  const cacheJSFilePrefix = “const CACHED_URLS = [\r\n ‘./'”;
  const cacheJSFileSuffix = “\r\n]\r\n”;
  const formatCacheJSFileEntry = (file) => “,\r\n ‘” + file + “‘”;

  const cacheFiles = glob.sync(pathPrefix + “**”, {nodir: true});
  let mappedCacheFiles = cacheFiles.map((file) => mapPathPrefix(mapPathDelimiters(file)));
  let cacheJSFile = cacheJSFilePrefix;
  mappedCacheFiles.forEach( file => cacheJSFile += formatCacheJSFileEntry(file));
  cacheJSFile += cacheJSFileSuffix;
  return cacheJSFile;
}

Comments Off on Progressive Web Apps – Dynamic Cache Script Creation

Tutorials/examples used

1. Simple tutorial here on free code camp
2. Building Progressive Web Apps by Tal Ater. This includes a source example on github.

Issues/Points Found:

1. My angular code has a number of cases where a query string is passed to a url. When caching a url or searching for one in the cache, it is important to remove all query strings and “#” fragment identifier. All my urls which are added to the cache are constants from a js/json file, and so no explicit removal is needed in the code. For searching, I use the {ignoreSearch: true} option on caches.match which strips all these on searching. Initially I appeared to have issues getting this working on all browsers, especially mobile, but in due course I established that there were other bugs at play, and in fact the above option works reliably across all browsers, including chrome and firefox on android.
2. A valid manifest .json must be declared in index.html, and this must have at least one icon of at least 144×144 in size. This is a PWA rule.
3. A promise was not being returned to the install event in free code camp example – this was due to braces being used in the install event code, but no explicit return was used to return the promise. The code did use a waitUntil in the install event, so clearly was expecting to wait for the cache additions to complete. The event waitUntil will therefore get an undefined return and not wait for the promise (i.e. the addAll to the cache) to complete. The PWAbook example above does do this correctly, and the code is very similar, but I copied and pasted code from each of the above in various examples and did not notice the missing return keyword in the free code camp example. This was subsequently remedied and all my code.
4. Service worker are always installed immediately after registration see here:
   I have not always seen this. I have often failed to see the console logs for the install event and the cache add/creation on the web site devtools console, but I suspect this is due to an existing service worker being used rather than a new one. The service worker has its own dev tools page and having just loaded a PWA web site locally on a new unused port, I noticed that the install event console logging was visible both on the dev tools for the pwa site and for the newly created service worker. All of this happened before any attempt was made to install the PWA app via the browser.
   Service workers are reused unless terminated/removed, even when the web app is updated. However you can force a new service worker on update by visiting the Application/service workers menu option on the Application page of the dev tools, and ticking update on reload. I did not prove this/try out in detail, but I did leave it on normally when developing.
   This issue may also be due to timing issues due to the bug in 1/, the promise not being returned therefore the code does not wait for all the urls to be added to the cache.
5. Service workers may be viewed via the following chrome urls:
   chrome://inspect/#service-workers
   chrome://serviceworker-internals/
6. For android, there are no dev tools available, but you can enable usb debugging. Visit the developer options under settings, and then enable usb debugging. You can then plug the phone into a windows pc via usb, and visit chrome://inspect/#devices to list the available usb devices, and connect to them, which opens a dev tools window.
7. Note that it takes up to a minute or so after the addAll is performed to the cache for it to appear as populated in the Applications tab of the chrome inspector. This was misleading at first as it looked as if it had failed/not been performed when it actually had.
8. Initially I did often see errors loading certain urls into the cache. To log this, I switched to using a loop to iterate the cache urls array, and used sequential cache.add() calls rather than cache.addAll(). This allowed logging explicitly of the ones in error (cache.addAll did not give detailed errors about the urls that had failed). Cache.addAll() is better though as it performs the adds in parallel and so is faster, so having got the pwa working I have reverted to using cache.addAll. I suspect these errors were due to code issues, in particular the above timing issues due to the missing return keyword.

Having battled through all these issues, I was able to get reliable operation, both online and offline, and both on windows and Android, with both Chrome and Firefox.

Comments Off on Progressive Web Apps – Technical Points/Issues Found

I started looking into PWAs as an alternative to the Ionic framework which I have used previously to prototype an offline/online mobile application for my mobile places guide.

My main issue with frameworks such as Ionic was the need for multiple application codebases – one admin application for web based use online (which could also be used on mobile when online), and another application for offline or online use on mobile when actually out in the sticks and looking for a local place that best meets my already specified favourite criteria.

The immediate attraction of a PWA was the simplification due to a single codebase framework. As it is browser based, it can run with any web framework/app architecture and be usable online, and installable for offline use. Some of the main benefits are as follows:

• Allows a single application framework such as Angular or React to be used for a web application which can be used online or installed for offline/online use just like a native app, i.e. can be run as an app from the home screen on a mobile. This allows a single application framework/codebase to be used for all scenarios. This was the key advantage for me – the need for multiple codebases/frameworks when developing an app as a small enterprise is a serious limitation. There is also the potential need for multiple offline applications/frameworks to support all the required platforms, so there can even be a need for maintaining multiple offline native app codebases.
• PWAs are widely supported across browsers/platforms and are a standard.
• A PWA greatly simplifies the installation process. One major issue with web sites which have a mobile app is the need to visit an app store and go through the install process before being able to continue. This really breaks the flow of use.
• Web sites are really keen for users to install their app, as this offers functionality such as offline working and push notifications even when the app is not active/not in use. However, they typically resort to an pattern known as a full page interstitial ad. This interrupts the user, often with a full page overlay, right when they are interacting with content on the web site, in an attempt to force them to install the app – commonly called the door slam antipattern. This takes a number of steps, followed by actually running the app and returning to their previous context. A survey by Google showed that a significant percentage of users just dump the app installation and their web site activity completely when presented with the door slam.
• PWAs have a much more seamless install process which is performed with a click directly within the browser. It does not involve visiting an app store, installing an app, and selecting options/enabling app permissions. Therefore significantly less users are put off by this.

Initial tryout/Proof of Concept

I branched the codebase for my existing Property Details application. This was a good example to try, as it has significant complexity:

• Web application using the Angular framework (v13 at the current time).
• PrimeNG component framework for angular is extensively used, including complex components such as the photo gallery
• Dynamic scripting/script switching for configuration and PrimeNG theme switching.
• Use of the angular APP_INITIALIZER to initialise/configure the app prior to angular bootstrapping, and use of angular/browser polyfills.
• Use of the Webpack module bundler to bundle and install the application

I felt that if I could get this application to run seamlessly as a PWA with minimal effort and changes to the existing code, this would be a valuable proof of concept to justify taking PWAs forward.

My conclusion was very positive – whilst I had a number of technical/learning issues to overcome, I succeeded in getting the property details app to run as a PWA both online and fully offline in Chrome, edge, and Firefox, on both Windows and Android, with minimal code changes. The only code interventions were to add the necessary support for PWAs, which were minimal. None of the existing code had to be modified. I just needed to add a json PWA manifest file, a couple of scripts, and an icon image. I then just needed to modify the index.html page to declare the manifest and call the first of the scripts. Going forward, I will need to try the other PWA features such as background/offline sync, push notifications, and service worker to page/Window to service worker communication. I will also need to make some effort to smooth the app rendering to bring up a page quickly, especially when offline, as the current POC is somewhat jerky on occasion, with some delays on rendering. However, it works reliably as expected.

I also have not looked into the database functionality re offline/online use and syncing. For the database, it is tempting to stick with CouchDB, as PouchDB is available as a wrapper for IndexedDB and could be used for background sync with a CouchDB as I originally planned with my Ionic POC. One issue which needs exploring on this is the availability of CouchDB hosting – my ISP, Zen Internet, use CPanel which does not host CouchdB. If this is an issue I would need to either use IndexedDB with e.g. MySQL at the back end and my own abstraction/syncing between the two, or use continue to use PouchDB still with a MySQL back end – this whole area needs investigation.

I have therefore decided to look into using PWAs further as a replacement for my existing Ionic offline POC/tryout.

Comments Off on Progressive Web Apps

I was looking for a good way to do this immediately following the issues I hit changing the username domain on Office 365.

After experimenting with doing large zip downloads to back up locally, which was intermittent with large downloads, I found this post here and I did the following, do force all OneDrive files to be synched locally all the time:-

• I unlinked OneDrive by visiting Account under settings and clicking “unlink this PC”. I then visited the OneDrive local folder root, which was under c:\Users\SteveW\OneDrive (for my free hotmail OneDrive) and  “c:\Users\SteveW\OneDrive – Salient Soft” for my Office 365 business account, and moved them along with all the files, to the correspponding directory on the D Drive. I then visited One Drive again in the system tray and re-added the account/username, picking the link to browse for a different folder, and browsed for the new folder. When I did this it warned that files were already present, which was fine as I had just moved them, so I told it to continue. This allowed me to move from the default drive to my preferred non-system drive which had more disk space.
• I again visited OneDrive in the system tray, and opened Settings, then selected Sync and backup on the left menu, and picked advanced settings
• I then visited Files on-demand (noting the help link which is the above post), and selected “Download all files”. Once I allowed this to synchronise fully, after a few hours, all the files were downloaded locally.
• I then just needed to do local backups as normal, as my local copies of the cloud would be fully up to date.

Comments Off on Backing Up Office 365 OneDrive files

As I was no longer trading via a limited company, I wanted to remove the “Ltd” references to my organisation and URLs in Office 365.

One key point was that the login username was SteveWoodley@SalientSoftLtd.onmicrosoft.com, and I wanted to remove the Ltd. Office 365 support advised on this and I did the following:

• I added a new domain as above without the Ltd. To do this, I visited the admin centre, clicked “Show All” on the left menu, and then dropped down the settings menu, again on the left. I added the new domain, and also clicked the button to make it the fallback domain.
• I then selected users under the active users menu, selected my user, and then clicked the link for “manage username and email”. I then clicked the pencil to edit the primary username and email, and selected the new domain that I had just entered above.
• I did this again for all the usernames.
• I noted that when I visited onedrive, the URL root showed as “https://salientsoftltd-my.sharepoint.com/”, i.e. it still had the “Ltd” in it. After searching online, this post here states that you cannot change this as it is the tenant domain name. It would mean creating a new tenant/Office 365 subscription and moving all the data, settings etc. from the old to the new. As this was not a big deal, and as the support staff did not see this as a problem, I left it alone, and was happy that at least the username was corrected.

After this was done, I found that onedrive could not see all the files and was concerned about lost files. I did note that under the user settings, there is a OneDrive menu which allows you to create a link. This in fact created a working link, but the main OneDrive link from the admin etc. continued not to work.

Support advised just waiting a day to allow the changes to be resynchronised correctly at the server end. I did this, and it did indeed work out fine with everything working.

I had also initially had issues with OneDrive for Windows 11 immediately after the change, regarding use of the modified username/domain, and could not see the files. Again, this all sorted itself out after a day or so and everything was fine.

Comments Off on Changing Office 365 to remove Ltd Company reference

This follows on from my previous post here.

Addition of a USB extender

As before, the mobile router performed around 4 times faster in the wooden garden office than in the brick build house, so it made sense to locate the router there when in use. However, I still wanted to allow shared internet around the house and on wifi when using the backup, and doing this another way e.g. by trying to share the connection from a windows PC in the garden office was not straightforward, and prevented the features of the Fritz box like its wifi from being used. I therefore obtained an EZCOO 50m USB extender for this, to allow the mobile router to still be connected to the Fritz Box in the house, but located in the garden office. When initially connecting the EE router to my TV PC via the USB extender, I received errors like “The last USB device you connected to this computer has malfunctioned and windows doesn’t recognize it”. Initially This appeared to be cable length related, as if it was a data corruption issue, as it failed on a full run to garden office with generous 7m flex cable at garden end. The full run was likely around the 50m max that was quoted by the EZCOO usb extender. However, this can also be caused by usb driver related issues. I tried to shorten the run by using the shortest cables I could each side of the Cat6 SWA run to the garden office. After this tidying up and retrying, the EE mobile router connected ok.

Significantly, I also received this error when testing the connection with my S22 Ultra mobile, again over the extender. However, when switching to using an older IOCELL Netdisk backup disk via its USB2 connection, it connected fine, which would tend to point more to driver or usb compatibility issues. Crucially, the Netdisk was able to copy significantly sized files both ways successfully over the full distance, indicating that the USB extender was working fine. Also and importantly, connecting the mobile router via the extender to the Fritz Box worked fine – the fritz box did not complain and always connected, but as always generally takes several minutes to switch over correctly between the primary and backup provider.

I continued to try file copying tests over the EE router connected to the TV PC via the extender, and these were all successful including 2-3GB copies to and from a OneDrive share. One issue that muddied the waters a lot was Windows file system caching of data both under Windows and with OneDrive – this led me to think copying had been completed when it hadn’t, or when a cache had been used to avoid the copy. Once I eliminated these issues all the tests were fine.

After retesting again today I had no problems connecting or speed testing, including tests done with the mobile router connected to the Fritz Box. I have had an occasional issue where speed test upload test did not work at all. I think that this may be a general ee mobile contention issue and perhaps if the upload link is too slow, ookla speed test just doesn’t bother and gives up.

DNS Issues

During the last couple of days, DNS has not been an issue with Zen overrides set, or by leaving the provider defaults. DNS still seems like it might be an issue, so it may be necessary to try another free public override on the DNS. The top rated free public servers are generally Cloud, Google, and OpenDNS, and details of these may be found on TechRadar here or on Broadband Genie here.

Points re topup and sim expiry

The terminology and process on this was not entire clear initially:-

• Firstly you need to purchase more credit, using a credit card, via ee.co.uk/topup . This can be done from any internet connection, by entering the mobile router phone number twice to authenticate (you don’t need a password to do this). The number was not listed in the paperwork for the router so I wrote it down on the paperwork for convenience.
• After you top up, i.e. buy credit, you need to visit add-on.ee.co.uk/purchase to get additional data using the credit you just purchased. Note that the word ‘purchase’ in the url here is ambiguous as you have already made a purchase with a credit card – you are just cashing in your credit to buy data. Note that in order to do this stage, your PC must be directly connected to the mobile router, i.e. you cannot be connected via the fritz box, even when it is in backup mode with the mobile router active. You get a choice of sizes, and you must have purchased enough credit previously to cover the size you want.
• In addition, it appears that there is a 6 month sim expiry issue for the PAYG sim used in the mobile router, as per here. You need to not just use the mobile router but also spend money every 6 months (180 days) or the SIM expires and you also lose any credit you had. You can phone EE to recover an expired SIM as detailed here, but if after 273 days you have done nothing, the sim and number expire completely and you will need a new sim and number. Therefore I decided my best option if not needing the backup for extended periods is to use it around twice a year before the 180 days expires, and buy the minimum minutes package (which is £5 at the time of writing).

Comments Off on EE Mobile Router Backup via Fritzbox 7530 – USB Extender and topup issues

Following on from my previous post on dynamic routing fundamentals, the ability to route dynamically across angular elements, i.e. to use the router to route to a dynamically loaded angular element, seems to be a sticking point.

Previously, I developed a web components example where a ‘fabric’ component was responsible for dynamically creating web components and displaying them. This did not use angular routing, but dynamically added them to the dom based on menu selections or url parameters. Subsequently, I wanted to investigate whether I could use angular routing to implement this instead.

I have found a number of examples of varying degrees of complexity, which go at least some of the way. However, I have not seen what I would term a reasonably straightforward, out-of-the-box solution, which does not involve too much deep diving into the internals of angular.

The following examples have been examined to date:

A complex dynamic one here with perhaps useful ideas but actual components still appear to be declared in the angular app:
https://medium.com/ngconf/dynamic-component-generation-in-lazy-loaded-routes-d2e0f9e89c57
The code is here:
https://github.com/theAlgorithmist/Angular8LazyLoadDynamic

Some specific ideas in this one here but still some way to go IMO…
https://medium.com/@timon.grassl/how-to-use-routing-in-angular-web-components-c6a76449cdb

This one perhaps looks the most promising, and may have all the relevant techniques needed, but It still appears to have statically declared component objects which are used dynamically in a dynamic order…
https://snyk.io/advisor/npm-package/angular-elements-router

Some more research is needed on this as I am keen to fine the simplest solution which uses as little as possible of the deeper features of angular.

Comments Off on Angular – Dynamic Routing between Angular Elements aka Web Components

One of my goals is to be able to dynamically create routes based on configuration data, typically loaded at pre-init time using an app initialiser.

Two approaches I have seen to doing this are as follows:

1. Using a custom Injector to use code to inject the routes.
   This is done in this example here. The code for it is available here.
2. Using Router.resetConfig() to change the routes after initially creating them.
   This is done in this example here. The code for it is available here.
   It is also done in this example here, in the app initialiser.

Both are dynamic, but method 1/ with the injector runs before any app initialiser, so would not be suitable for my needs (I verified this with a simple test with console logging). If this were used, any dynamic config would need to be fetched within the injector code or called directly from it. Method 2/ can be used after the app initialiser – I tried calling it successfully from the ngOnInit lifecycle hook in the app component which worked fine.

This example here uses method 1/, and the code for it is on github here. I downloaded the code and modified the original version, as it contains errors in the time of day checking code. I then saved the project to bitbucket. I branched it and modified it again to use resetConfig as above from ngOnInit, and removed the injector mechanism. This also worked fine.

Whilst these approaches allow dynamic creation of the routes, they assume that the routed components are known to the application or declared in it in some form. The basis of the standard approach to routing is that the component class referred to in the route is looked up, and its selector is extracted and used to create the component tag dynamically, inserting it after the appropriate <router-outlet>. This would not work in the case of angular elements, i.e. W3C web components. These are entirely dynamic and my examples which use these in the past have the element name/component tag defined dynamically as part of the config, along with the names of the scripts to be loaded to create the web component. Another approach will need to be investigated to look into routing of angular elements.

Comments Off on Angular Dynamic Routing

These points were found whilst trying some angular router examples.

Examples tried were these. Note that they are all on Stackblitx, and therefore zips can be downloaded.
It is not obvious how, but just make sure the project panel is open by clicking on the 2-page icon near the top left. Then click on the small cloud icon containing the down arrow, on the RHS of the project panel title.

1. https://www.tektutorialshub.com/angular/angular-routing-navigation/
2. https://stackblitz.com/edit/angular-ivy-spsz1g?file=src%2Fapp%2Fapp.component.ts
   Note that this one above is angular 15 so pretty up to date

1. Basic one with just 2 router states/2 buttons
   https://angular.io/guide/router-tutorial
2. Full Tour of Heroes tutorial
   https://angular.io/guide/router-tutorial-toh 
   This above one is angular 16 so latest

1. Routes are defined using RouterModule.forRoot. In some examples the route definitions/routes array is defined in app.module.ts (as in the basic angular.io example above). More typically the routes array is defined in a separate file to separate the concerns, such as app-routing.module.ts or app.routes.ts. It can then either be exported from this file and referenced in app.module.ts via RouterModule.forRoot (as in the tektutorials example above), or RouterModule.forRoot is itself also in the separate file defining the routes (as in the full Tour of Heroes example above, in app-routing.module.ts).
   
   
2. In all the examples, routing is done by creating a component and placing it as a sibling of the <router-outlet>tag, immediately after <router-outlet>. The created component is never placed as a child of the <router-outlet>. As per the full Heroes tutorial, <router-outlets> can be nested where required – a component activated from a <router-outlet> can then use other routes to activate further child components etc.
   
   
3. When refreshing e.g. a child page on zen apache or http-server, I got 404 errors. It is necessary to tell the server to redirect to e.g. index.html or whatever you are using, to get the routing to work. This was not the case when using “ng serve” as this takes care of the issue for you automatically. The same problem happens if you enter a url manually or from a link, to go to a detail child route page. This angular.io deployment page explains how to do this for various servers, including apache as used by Zen in my case. In my case I tested on Zen and needed to edit .htaccess as per the instructions. For this to work, I had to change the path in the instructions to the correct subdomain index.html file or it still failed. I also created a child .htaccess file in the subdomain folder where my app was hosted. This ran in addition to the main .htaccess, and it all worked fine. I did notice that the lowest child hero pages on the full tour of heroes example above still failed with a 404. However, they also failed with a 404 when served via “ng serve”, so I concluded that this was a coding issue in the example and did not investigate further. My .htaccess for the full heroes example under zen was as follows:-

RewriteEngine on

RewriteOptions inherit

# BEGIN Angular Routing

RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -f [OR] 
RewriteCond %{DOCUMENT_ROOT}%{REQUEST_URI} -d 
RewriteRule ^ - [L] 

# If the requested resource doesn't exist, use index.html 
RewriteRule ^ /angular-router/angular-full-heroes-example/index.html

# END Angular Routing

When a route is activated, its target component is always recreated and any previous component for the corresponding <router-outlet> is destroyed. I confirmed this with all the above examples by logging in the ngOnInit and ngOnDestroy life cycle methods. Components are not switched in and out by showing and hiding.

Comments Off on Angular Routing