February 6th, 2024
1:52 pm
Progressive Web Apps – Dynamic Cache Script Creation

Posted under Angular & Knowledge Base & PWAs & Web
Tags ,

My PWAs dynamically create and load a url cache in the service worker, when the installation event fires.
As this cache contains all the files in the dist/project-name directory tree (i.e. the entire deployment) for an angular app, I was keen to create the script containing the list of files to be cached dynamically as a post build step, so that the post build script could be generic and adaptive to the build contents.

This turned out to be pretty straightforward. The following points are noteworthy:

  1. I wanted to dynamically create a JS file which defined an array containing the urls of all the files to be cached. This script would then be included by the service worker to be used when it created and loaded the cache, during its install event.
  2. The above JS file was imported by the service worker script. I had some issues getting this to work via modules/module support, and noted from this post here that using the importScripts function instead (e.g. importScripts(‘pwa-cache-urls.js’);) gave better cross browser support/backwards compatibility. I therefore switched to using importScripts and it all worked fine.
  3. glob.sync was used to generate an array containing all the deployed files to be cached, excluding directories using {nodir: true}. This was then mapped/formatted as required into a string containing all the script for the file.
  4. fs.writeFileSync was used to create the script file from the above string.
  5. The project name and the generated script file name were passed as arguments to the post build file, in package.json. Whilst I did look for a dynamic way to derive/lookup the project name, an easy way to do this was not obvious and there was no loss of flexibility in just passing it in the postbuild script definition in package.json, along with the name of the dynamically created script file.

This all worked fine, and I was able to create a generic post build script that would be able to discover the files to be cached dynamically and create the script for them, for any angular project I was working with.

The prototype example code for the post build is below:

const glob= require(‘glob’);
const fs = require(‘fs’);

const args = process.argv;
validateArgs(args);
const projectName = args[2];
const cacheJSFileName = args[3];

const pathPrefix = ‘dist/’ + projectName + “/”;
const cacheJSFilePath = pathPrefix + cacheJSFileName;

fs.writeFileSync(cacheJSFilePath, buildCacheJSFile());

function validateArgs(args) {
  const argErrors = [
    “Project Name and Cache Filename were not passed to post-build.js”,
    “Cache Filename was not passed to post-build.js”
  ]
  if (args.length < 4) {
  throw new Error(argErrors[args.length – 2]);
  }
}

function buildCacheJSFile() {
  const mapPathDelimiters = (path) => path.replaceAll(“\\”, “/”);
  const mapPathPrefix = (path) => path.replace(pathPrefix, “”);

  const cacheJSFilePrefix = “const CACHED_URLS = [\r\n ‘./'”;
  const cacheJSFileSuffix = “\r\n]\r\n”;
  const formatCacheJSFileEntry = (file) => “,\r\n ‘” + file + “‘”;

  const cacheFiles = glob.sync(pathPrefix + “**”, {nodir: true});
  let mappedCacheFiles = cacheFiles.map((file) => mapPathPrefix(mapPathDelimiters(file)));
  let cacheJSFile = cacheJSFilePrefix;
  mappedCacheFiles.forEach( file => cacheJSFile += formatCacheJSFileEntry(file));
  cacheJSFile += cacheJSFileSuffix;
  return cacheJSFile;
}

Comments Off on Progressive Web Apps – Dynamic Cache Script Creation

February 2nd, 2024
11:43 am
WordPress post content has disappeared when trying to edit existing post

Posted under Knowledge Base & Web & Wordpress
Tags , ,

I just hit this when editing a couple of existing posts.

All the post content except the header had disappeared, and I did not seem to be able to recover it. Whilst trying, I did receive an error once about the block format, indicating there was a bug with the block layout additions in WordPress. It is possible that there is also an issue with an existing plugin/plugin version that I am using.

In the end, I discovered a reasonable workaround. When editing the post to update it, click the triple dot menu in the top RHS of the right hand pane. Then select the code editor rather than the visual editor. This revealed all the existing content but in raw code mode.

When I then clicked on the Visual editor again, all of the content came back as originally posted.

I looked for solutions/workarounds online but could not find anything concrete. Whilst this is a nuisance, I can live with the workaround for now, until I can find the time to give more effort to a proper fix/solution.

Comments Off on WordPress post content has disappeared when trying to edit existing post

February 2nd, 2024
11:14 am
Progressive Web Apps

Posted under Angular & Knowledge Base & PWAs & Web
Tags , , ,

I started looking into PWAs as an alternative to the Ionic framework which I have used previously to prototype an offline/online mobile application for my mobile places guide.

My main issue with frameworks such as Ionic was the need for multiple application codebases – one admin application for web based use online (which could also be used on mobile when online), and another application for offline or online use on mobile when actually out in the sticks and looking for a local place that best meets my already specified favourite criteria.

The immediate attraction of a PWA was the simplification due to a single codebase framework. As it is browser based, it can run with any web framework/app architecture and be usable online, and installable for offline use. Some of the main benefits are as follows:

  • Allows a single application framework such as Angular or React to be used for a web application which can be used online or installed for offline/online use just like a native app, i.e. can be run as an app from the home screen on a mobile. This allows a single application framework/codebase to be used for all scenarios. This was the key advantage for me – the need for multiple codebases/frameworks when developing an app as a small enterprise is a serious limitation. There is also the potential need for multiple offline applications/frameworks to support all the required platforms, so there can even be a need for maintaining multiple offline native app codebases.
  • PWAs are widely supported across browsers/platforms and are a standard.
  • A PWA greatly simplifies the installation process. One major issue with web sites which have a mobile app is the need to visit an app store and go through the install process before being able to continue. This really breaks the flow of use.
  • Web sites are really keen for users to install their app, as this offers functionality such as offline working and push notifications even when the app is not active/not in use. However, they typically resort to an pattern known as a full page interstitial ad. This interrupts the user, often with a full page overlay, right when they are interacting with content on the web site, in an attempt to force them to install the app – commonly called the door slam antipattern. This takes a number of steps, followed by actually running the app and returning to their previous context. A survey by Google showed that a significant percentage of users just dump the app installation and their web site activity completely when presented with the door slam.
  • PWAs have a much more seamless install process which is performed with a click directly within the browser. It does not involve visiting an app store, installing an app, and selecting options/enabling app permissions. Therefore significantly less users are put off by this.

 

Initial tryout/Proof of Concept

I branched the codebase for my existing Property Details application. This was a good example to try, as it has significant complexity:

  • Web application using the Angular framework (v13 at the current time).
  • PrimeNG component framework for angular is extensively used, including complex components such as the photo gallery
  • Dynamic scripting/script switching for configuration and PrimeNG theme switching.
  • Use of the angular APP_INITIALIZER to initialise/configure the app prior to angular bootstrapping, and use of angular/browser polyfills.
  • Use of the Webpack module bundler to bundle and install the application

I felt that if I could get this application to run seamlessly as a PWA with minimal effort and changes to the existing code, this would be a valuable proof of concept to justify taking PWAs forward.

My conclusion was very positive – whilst I had a number of technical/learning issues to overcome, I succeeded in getting the property details app to run as a PWA both online and fully offline in Chrome, edge, and Firefox, on both Windows and Android, with minimal code changes. The only code interventions were to add the necessary support for PWAs, which were minimal. None of the existing code had to be modified. I just needed to add a json PWA manifest file, a couple of scripts, and an icon image. I then just needed to modify the index.html page to declare the manifest and call the first of the scripts. Going forward, I will need to try the other PWA features such as background/offline sync, push notifications, and service worker to page/Window to service worker communication. I will also need to make some effort to smooth the app rendering to bring up a page quickly, especially when offline, as the current POC is somewhat jerky on occasion, with some delays on rendering. However, it works reliably as expected.

I also have not looked into the database functionality re offline/online use and syncing. For the database, it is tempting to stick with CouchDB, as PouchDB is available as a wrapper for IndexedDB and could be used for background sync with a CouchDB as I originally planned with my Ionic POC. One issue which needs exploring on this is the availability of CouchDB hosting – my ISP, Zen Internet, use CPanel which does not host CouchdB. If this is an issue I would need to either use IndexedDB with e.g. MySQL at the back end and my own abstraction/syncing between the two, or use continue to use PouchDB still with a MySQL back end – this whole area needs investigation.

I have therefore decided to look into using PWAs further as a replacement for my existing Ionic offline POC/tryout.

Comments Off on Progressive Web Apps

October 9th, 2023
5:50 pm
Backing Up Office 365 OneDrive files

Posted under Knowledge Base & MS Office & Windows & Windows 11
Tags , ,

I was looking for a good way to do this immediately following the issues I hit changing the username domain on Office 365.

After experimenting with doing large zip downloads to back up locally, which was intermittent with large downloads, I found this post here and I did the following, do force all OneDrive files to be synched locally all the time:-

  • I unlinked OneDrive by visiting Account under settings and clicking “unlink this PC”. I then visited the OneDrive local folder root, which was under c:\Users\SteveW\OneDrive (for my free hotmail OneDrive) and  “c:\Users\SteveW\OneDrive – Salient Soft” for my Office 365 business account, and moved them along with all the files, to the correspponding directory on the D Drive. I then visited One Drive again in the system tray and re-added the account/username, picking the link to browse for a different folder, and browsed for the new folder. When I did this it warned that files were already present, which was fine as I had just moved them, so I told it to continue. This allowed me to move from the default drive to my preferred non-system drive which had more disk space.
  • I again visited OneDrive in the system tray, and opened Settings, then selected Sync and backup on the left menu, and picked advanced settings
  • I then visited Files on-demand (noting the help link which is the above post), and selected “Download all files”. Once I allowed this to synchronise fully, after a few hours, all the files were downloaded locally.
  • I then just needed to do local backups as normal, as my local copies of the cloud would be fully up to date.

Comments Off on Backing Up Office 365 OneDrive files

October 9th, 2023
5:31 pm
Changing Office 365 to remove Ltd Company reference

Posted under Knowledge Base & MS Office & Windows & Windows 11
Tags , ,

As I was no longer trading via a limited company, I wanted to remove the “Ltd” references to my organisation and URLs in Office 365.

One key point was that the login username was SteveWoodley@SalientSoftLtd.onmicrosoft.com, and I wanted to remove the Ltd. Office 365 support advised on this and I did the following:

  • I added a new domain as above without the Ltd. To do this, I visited the admin centre, clicked “Show All” on the left menu, and then dropped down the settings menu, again on the left. I added the new domain, and also clicked the button to make it the fallback domain.
  • I then selected users under the active users menu, selected my user, and then clicked the link for “manage username and email”. I then clicked the pencil to edit the primary username and email, and selected the new domain that I had just entered above.
  • I did this again for all the usernames.
  • I noted that when I visited onedrive, the URL root showed as “https://salientsoftltd-my.sharepoint.com/”, i.e. it still had the “Ltd” in it. After searching online, this post here states that you cannot change this as it is the tenant domain name. It would mean creating a new tenant/Office 365 subscription and moving all the data, settings etc. from the old to the new. As this was not a big deal, and as the support staff did not see this as a problem, I left it alone, and was happy that at least the username was corrected.

After this was done, I found that onedrive could not see all the files and was concerned about lost files. I did note that under the user settings, there is a OneDrive menu which allows you to create a link. This in fact created a working link, but the main OneDrive link from the admin etc. continued not to work.

Support advised just waiting a day to allow the changes to be resynchronised correctly at the server end. I did this, and it did indeed work out fine with everything working.

I had also initially had issues with OneDrive for Windows 11 immediately after the change, regarding use of the modified username/domain, and could not see the files. Again, this all sorted itself out after a day or so and everything was fine.

Comments Off on Changing Office 365 to remove Ltd Company reference

September 6th, 2023
1:15 pm
EE Mobile Router Backup via Fritzbox 7530 – USB Extender and topup issues

Posted under Hardware & Knowledge Base & Networks & PC
Tags , ,

This follows on from my previous post here.

Addition of a USB extender

As before, the mobile router performed around 4 times faster in the wooden garden office than in the brick build house, so it made sense to locate the router there when in use. However, I still wanted to allow shared internet around the house and on wifi when using the backup, and doing this another way e.g. by trying to share the connection from a windows PC in the garden office was not straightforward, and prevented the features of the Fritz box like its wifi from being used. I therefore obtained an EZCOO 50m USB extender for this, to allow the mobile router to still be connected to the Fritz Box in the house, but located in the garden office. When initially connecting the EE router to my TV PC via the USB extender, I received errors like “The last USB device you connected to this computer has malfunctioned and windows doesn’t recognize it”. Initially This appeared to be cable length related, as if it was a data corruption issue, as it failed on a full run to garden office with generous 7m flex cable at garden end. The full run was likely around the 50m max that was quoted by the EZCOO usb extender. However, this can also be caused by usb driver related issues. I tried to shorten the run by using the shortest cables I could each side of the Cat6 SWA run to the garden office. After this tidying up and retrying, the EE mobile router connected ok.

Significantly, I also received this error when testing the connection with my S22 Ultra mobile, again over the extender. However, when switching to using an older IOCELL Netdisk backup disk via its USB2 connection, it connected fine, which would tend to point more to driver or usb compatibility issues. Crucially, the Netdisk was able to copy significantly sized files both ways successfully over the full distance, indicating that the USB extender was working fine. Also and importantly, connecting the mobile router via the extender to the Fritz Box worked fine – the fritz box did not complain and always connected, but as always generally takes several minutes to switch over correctly between the primary and backup provider.

I continued to try file copying tests over the EE router connected to the TV PC via the extender, and these were all successful including 2-3GB copies to and from a OneDrive share. One issue that muddied the waters a lot was Windows file system caching of data both under Windows and with OneDrive – this led me to think copying had been completed when it hadn’t, or when a cache had been used to avoid the copy. Once I eliminated these issues all the tests were fine.

After retesting again today I had no problems connecting or speed testing, including tests done with the mobile router connected to the Fritz Box. I have had an occasional issue where speed test upload test did not work at all. I think that this may be a general ee mobile contention issue and perhaps if the upload link is too slow, ookla speed test just doesn’t bother and gives up.

DNS Issues

During the last couple of days, DNS has not been an issue with Zen overrides set, or by leaving the provider defaults. DNS still seems like it might be an issue, so it may be necessary to try another free public override on the DNS. The top rated free public servers are generally Cloud, Google, and OpenDNS, and details of these may be found on TechRadar here or on Broadband Genie here.

Points re topup and sim expiry

The terminology and process on this was not entire clear initially:-

  • Firstly you need to purchase more credit, using a credit card, via ee.co.uk/topup . This can be done from any internet connection, by entering the mobile router phone number twice to authenticate (you don’t need a password to do this). The number was not listed in the paperwork for the router so I wrote it down on the paperwork for convenience.
  • After you top up, i.e. buy credit, you need to visit add-on.ee.co.uk/purchase to get additional data using the credit you just purchased. Note that the word ‘purchase’ in the url here is ambiguous as you have already made a purchase with a credit card – you are just cashing in your credit to buy data. Note that in order to do this stage, your PC must be directly connected to the mobile router, i.e. you cannot be connected via the fritz box, even when it is in backup mode with the mobile router active. You get a choice of sizes, and you must have purchased enough credit previously to cover the size you want.
  • In addition, it appears that there is a 6 month sim expiry issue for the PAYG sim used in the mobile router, as per here. You need to not just use the mobile router but also spend money every 6 months (180 days) or the SIM expires and you also lose any credit you had. You can phone EE to recover an expired SIM as detailed here, but if after 273 days you have done nothing, the sim and number expire completely and you will need a new sim and number. Therefore I decided my best option if not needing the backup for extended periods is to use it around twice a year before the 180 days expires, and buy the minimum minutes package (which is £5 at the time of writing).

Comments Off on EE Mobile Router Backup via Fritzbox 7530 – USB Extender and topup issues

July 18th, 2023
4:56 pm
Surface Pro 4 Keyboard not working

Posted under Hardware & Knowledge Base & PC & Windows & Windows 10
Tags ,

We have had this issue a number of times, where the cover keyboard just won’t work, even after rebooting several times.

This appears to be a common problem which is griped about online, and the solutions are varied, not that clear/consistent, and do not always work. I even replaced the cover keyboard, which did fix the problem at the time, but I then later found that the original one still worked!

IMO this looks like some kind of underlying bios problem.

After trying a number of ideas which did not work, including removing the keyboard driver, and disabling/re-enabling the keyboard in the UEFI bios, I found that booting into the UEFI bios (hold +volume button whilst pressing on button), and then rebooting again, seemed to work.

This site here appeared to have some helpful ideas, including running the keyboard troubleshooter, which I had not seen previously. As the troubleshooter initially said that there were no problems (!), I had to run additional troubleshooters and then select keyboard, as per the post, for it to work. In my case, the troubleshooter ran fine, but the keyboard was already working, so I’ll have to wait until next time to see if this helps to solve the problem, should it occur again.

Comments Off on Surface Pro 4 Keyboard not working

April 13th, 2023
4:59 pm
EE Mobile Router Backup via Fritzbox 7530

Posted under Hardware & Knowledge Base & Networks & PC
Tags , ,

I obtained an EE mobile router as backup for the occasions when my main FTTC internet goes down. The main internet went down recently for and extended period due to an OpenReach Fibre connection issue to the exchange.

EE have the best mobile data coverage for my area per the ofcom coverage site here.

Zen internet sent me an EE mobile router as backup during the above outage (I presume they knew that EE had the best coverage in my area).

I was able to readily plug the router into a PC and get mobile coverage.

However, doing this via my Fritzbox 7530 needed some tweaking. The fritzbox has mobile internet backup capability via a submenu on its internet menu. This allows enabling the mobile router (plugged in via its usb port) as a backup only when the main internet is down.

Initially when I tried this having unplugged the main DSL connection, the internet showed as up, but DNS translation did not work. I found that under the internet menu/Account Information/DNS Server settings, I had overridden the DNSV4  servers provided by the ISP with specific Zen ones. Once I returned this setting to “use DNSv4 servers assigned by the internet service provider (recommended)”, DNS then worked on the mobile connection. In addition, automatic failover to mobile worked correctly when I unplugged the DSL connection, although this did take a few minutes to stabilise and was not instant.

In practice, I planned not to leave the mobile router plugged in, but to connect when needed. However, the fact that it is capable of providing shared internet and wifi via the router to my whole network was a real plus.

I found that the EE mobile router performed particularly well from my garden office (which is of wooden construction). I obtained approx 50Mb down and 1 Mb up in this situation – the down link speed was actually better than my FTTC which is around 32Mb. However in the house via the Fritzbox 7530, this reduced to around 20Mb down, presumably due to the brick construction blocking the signal more than the wooden building did.

However, in both of these situations the mobile data rate was way better than I had previously obtained by tethering my Vodafone Mobile – in the latter case, the connection was very slow and intermittent – sometimes I could not get a connection at all, so it would have been no use as a backup for remote working. The EE mobile router was perfectly suited for use as a remote working backup, although I planned not to leave it plugged in permanently as a hot backup but to manually plug in when required, as it was not instant anyway, and in practice I may have to buy additional minutes from EE as they seem to expire if you don’t use them within a set period.

 

Comments Off on EE Mobile Router Backup via Fritzbox 7530

October 31st, 2022
5:39 pm
Editing WMF files in Office – Word/Powerpoint

Posted under Knowledge Base & MS Office & Powerpoint
Tags

I had an existing logo which was created in CorelDraw, and exported to Windows Metafile (WMF) format.

This can be imported into office directly. Having done this, if desired, the graphic can be typically ungrouped into its constituent graphic elements via its context menu (right click/group/ungroup). Having done this, you can then tweak the individual elements e.g. change colours/fonts used etc. I tried this in both Word and Powerpoint, and had done this previously in an existing powerpoint presentation.

Details of the WMF format and usage may be found on Wikipedia here.

Comments Off on Editing WMF files in Office – Word/Powerpoint

October 5th, 2022
4:16 pm
Using Spring Data JPA with Oracle

Posted under CouchDB & Java & JPA & Knowledge Base & Oracle & Spring & Spring Boot & Spring Data JPA
Tags , ,

I was using Spring Data JPA with Oracle as part of an export process from CouchDB to Oracle. The CouchDB export application first exports my existing Places data from CouchDB, then maps it to the entity model I am using for JPA/Oracle, then persists it into Oracle, and finally performs some rudimentary listing of the Oracle data by logging it.

Some issues/points came to light during this excercise which I am noting here:-

1/ My connect string to Oracle was using a Pluggable database in Oracle XE. This needs to use a service name (xepdb1) rather than a SID (xe) for the connection. To do this I needed to use a slash before the service name in the application properties, to indicate a service name, rather than the colon I would normally use for a SID:

# Oracle settings
spring.datasource.url = jdbc:oracle:thin:@localhost:1521/xepdb1
spring.datasource.username = places
spring.datasource.password = xxxxxxxx

2/ I changed the Hibernate naming strategy in the application properties to prevent unwanted snake casing of Oracle column names:

# Hibernate settings
spring.jpa.hibernate.naming.physical-strategy = org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl

3/ For persisting to Oracle, I had the choice of using Spring Data JPA, calling save() or saveAll() on the Individual Repository interfaces that extended CrudRepository (such as PlacesRepository), or I could use the Entity Manager. The Spring Data methods would persist or update fine, and could also save an iterable in one call. However, I had some small additional logic as I was passing and returning a Map. As I chose not to put this logic in the service layer, I therefore used an ordinary JPA Dao (repository) class, and autowired the entity manager via the @PersistenceContext annotation. I could therefore incorporate my logic in a single call and use entityManager.merge() to persist. I felt that situations vary and that it was not always appropriate to use Spring Data Interfaces for some use cases. However the Spring Data find queries were useful and clever for querying the database as they allowed for example a find by key with a like, and order by, just by declaring the appropriate method calls in the interface. I also used the @Query annotation for a custom query, which again was just annotated on a method in the interface.

4/ I had issues whereby the foreign keys on dependant entities were not being auto populated and threw up as null, even when I set nullable=false on the @JoinColumn annotations. In the end it was a simple case of not populating the objects in my model on both sides of the join before persisting – i.e. I was populating a list in a parent object for a one-to-many, but not populating the parent in the dependant object. Doing this resolved the problem and everything persisted automatically with a single parent persist/merge, with hibernate allocating primary keys via Oracle sequences, and then auto populating them in the foreign keys. This stack overflow post here pointed me in the right direction on this. I kept the nullable=false annotations where appropriate to correctly match the relationship I was using, and what was in Oracle.

5/ Once I was at the point of listing everything I had persisted, using Spring Data queries and some simple logging, I was initially hit with Hibernate multiple bag fetch exceptions when listing the data. At this stage I had annotated my repository methods as @Transactional rather than the service layer, and was therefore using explicit eager loading annotations as I needed everything to load before the methods returned. This post here details the issue, and it appears that Hibernate can only eagerly load one collection at a time. My solution to this was to take the more normal route of annotating my service methods as @Transactional rather than the repository, which then allowed a number of repository calls to be made from a single service method, all in the same transaction. Whilst this did expose the service layer to the transaction architecture and javax.transaction package, it is the more normal approach and gives flexibility in the service layer. In my case I could then just revert to the default lazy loading and perform the logging in the same service layer method (which I was happy to do as this was a very basic example just to demonstrate fetching the imported data from Oracle). Everything then worked fine.

 

Comments Off on Using Spring Data JPA with Oracle