Joomla 4 – Joomla API

by Marc Dechèvre

woluweb

Marc

Present slides https://slides.woluweb.be

At the end of this presentation, you will be able a.o. to import your Articles together with their Custom Fields to your Joomla website directly from a Google Sheet thanks to the Joomla API

This presentation was made

1. Please contribute

Help improving this presentation that will be converted to a Joomla Community Magazine article:

then please get back to me! ❤️💙🧡💚

1. How this presentation came to life

There have already been several presentations about the Joomla API.

They were certainly interesting (showing Postman, explaining how you can create a Smartphone App exchanging with your Joomla website, …) but I was always a bit frustrated because :

This changed on 23 September 2022 where I had the chance to be present at JoomlaDay D-A-CH (gathering several german-speaking countries, namely Germany – Austria – Switzerland). In particular Peter Martin gave an excellent “How to use APIs in your Joomla website?” (see links hereafter): At the end of his session, Peter would give a few little PHP scripts showing how to create or update Articles.

I asked to Peter whether he could share his scripts. He did even better: there is a new official website Joomla! Programmers Documentation with a page about Web Services and Peter made a Pull Request on Github to add the examples (see hereafter).

Coming back to Belgium, I was really excited to play with these examples.

Then I contacted Alexandre Elisé from Martinique, who is very passionate about the Joomla API and we elaborated on these examples. Indeed

So after several iterations over a few weeks, we ended up with a series of “little” scripts. The goal was indeed

So a long story short: this presentation aims at allowing a maximum of people to start using the Joomla API, which offers a strong competitive advantage compared to other CMS for example.

2. Resources

Peter Martin – “How to use APIs in your Joomla website?” at JoomlaDay D-A-CH on 2022.11.23

Alexandre Elisé

When you go to https://github.com/alexandreelise/j4x-api-examples please click on the Star button if you appreciate Alexandre’s scripts & contribution : Alexandre

Joomla! Programmers Documentation (in construction)

Joomla! Documentation

3. How to get the Joomla API Token

Before using the Joomla API (and any of the scripts below) we need a Joomla API Token.

At the moment, the Joomla API Token can only be generated for Super Users.

So be sure to keep it secret because any person having somehow access to that Token is de facto a Super User and can do everything on your website, including deleting everything.

Note: as far as I know, it is the idea at some point to allow non Super Users to access the API (which is necessary if you want to give a Token to a third-party who only wants to fetch data for example).

[EDIT] On 2022.11.25 Alexandre shares this additional information on Facebook:
“You don’t need to be a Super User to access the Joomla Api. You can create a Web Services usergroup for example, go to Global Configuration and give this Group Permissions for API and site login to be able to see the token. You need also to add the Web Services group you just created to the list of allowed groups in the token user plugin config. But for now you still cannot have fine-grained ACL and Access Level per endpoint.For more security you could also disable sensitive webservices plugins like the users web services plugin but then the consumer app using the token cannot manage user info. I haven’t tried it yet, but I think if you give specific permissions to the Web Services usergroup then add only app user in this group you could achieve want you want even if it’s still not per route / endpoint.”
“Just did a quick test on a fresh joomla 4.2.5 install you don’t need to be Super User. morever Joomla seems to already handle permissions per endpoint but only for core endpoints. At least that’s what I’ve tried. that means that if you create for example a web services group to manage all consumers of your Joomla web services and then deny create patch delete users for this usergroup the app/consumer of the joomla web services will get a 403 Forbidden http status code when trying to perform denied actions.”
Joomla Api Without Super User > Demo video (18 min)

3.1. Create a new User

Actually you could of course create a Joomla API Token for your own account.

But it is probably a good practice to create a distinct user

joomla-api

3.2. Give a strong password

joomla-api

3.3. Assign to the User Group Super User

joomla-api

3.4. Go the Joomla API Token tab and Read

joomla-api

3.5. Save

Now that you have saved the User, the Joomla API Token has been created… but for security reasons only that given user can see it (nobody else, not even other Super Users).

joomla-api

3.6. Log in as the new User and copy the Token

joomla-api

4. Create, Update, Fetch and Delete Joomla Articles with the API

4.1. Create Joomla Articles with the API

4.1.1. Terminology

Some basic terminology first: When using the Joomla API

4.1.2. See the Articles Manager

This is a fresh website so I see that “No Articles have been created yet”

joomla-api

4.1.3. Prepare the POST script

Create your POST script on your website

Customize your POST script on your website

joomla-api

4.1.4. articletext vs introtext and fulltext

Good to know: when you Create an article

4.1.5. The 4 necessary fields

To create an article

4.1.6. Execute your script

Simply

joomla-api

4.1.7. Go back to or refresh the Articles Manager

Tadam you can see that the Article was indeed created!

joomla-api

4.2. Update Joomla Articles with the API

4.2.1. Prepare the PATCH script

The procedure is totally similar to the one of the previous script:

Create your PATCH script on your website

Customize your PATCH script on your website

joomla-api

Good to know: when you Update an article

4.2.2. Execute your script

Simply execute your script as already explained above.

joomla-api

4.2.3. Go back to or refresh the Articles Manager

When opening the Article, we can indeed check that

joomla-api

4.3. Playing with Custom Fields

Adding a value to some Custom Field of an Article directly in the database using a query in PHPMyAdmin is not so obvious because it involves several Tables which are linked.

But as you will see, adding a value to some Custom Field of an Article via the Joomla API is super easy.

4.3.1. Create a Custom Field

In this example I create a Custom Field of Type “Text”

joomla-api

4.3.2. Adapt the PATCH script

Now I need to edit my api-patch script as follows

joomla-api

4.3.3. Execute your script

Simply execute your script as already explained above.

You already see on what is displayed that “temperature” is mentioned.

joomla-api

4.3.4. Go back to or refresh the Articles Manager

When opening the Article, we can indeed check that

joomla-api

4.4. Fetch Joomla Articles with the API

A good usecase for this script: an intranet or an app where one wants to display some news (articles) from your Joomla website

4.4.1. Prepare the GET script

Create your GET script on your website

Customize your GET script on your website

joomla-api

4.4.2. Bonus 1: select another Category

In this example, we suppose we want to get all articles from Category 2

Of course, you can easily change the ID of the Category you need by adapting the following line of code:

$categoryId = 2;

4.4.3. Bonus 2: only get Published Articles

What is you only want to get Articles which are Published.

In the database, you probably know that technically being

So all we need to change in the script is to change

/content/articles?filter[category]=

into

/content/articles?filter[state]=1&filter[category]=

4.4.4. Execute the script

Simply execute your script as already explained above.

joomla-api

4.5. Delete Joomla Articles with the API

Create your DELETE script on your website

Customize your DELETE script on your website

We don’t provide screenshots here. By now you know how it works!

5. How to launch any of the scripts automatically

Given the fact that our scripts here are independent of Joomla

Lauching any of the scripts can be simply done:

So obviously it means that you can automate the execution of the creation / update of your Joomla Articles at any time interval.

5.1. Example 1: via a nice button in the backend

In Joomla 4 it has become even easier than before to create a Custom HTML Module in the backend.

Take advantage of this feature to create a nice interface to your Users.

joomla-api

5.2. Example 2: via Joomla Task Scheduler

The Task Scheduler is a new feature introduced with Joomla 4.

joomla-api

6. Be the only one who can run your scripts

If somebody know that you have a script called for example api-test.php, this person could flood your websites with request on that url.

A typical way to avoir that is on Apache servers is to create a htpsswd, meaning you have to

When a htpsswd is enabled, you get a popup asking for the defined username & password before accessing.

You can typically

But we need to be more specific because

  1. we only want to block our scripts, not our whole website.
  2. we only want to block our scripts if they are launched in the browser, not if they are launched by the website itself
    • either via some code
    • either via the Task Scheduler

Let us first show a good example of such an .htaccess rule:

<FilesMatch "^api*">
AuthType Basic
AuthBasicProvider file
AuthUserFile /YOUR_PATH/public_html/.htpasswd
AuthName secure
<RequireAny>
Require valid-user
Require ip 127.0.0.1 185.221.181.208
</RequireAny>
</FilesMatch>

7. Change the API Token if needed

If you have any reason to think that your API key is not private any more (like me after having shared this presentation) then simply Edit the User in question, go to the Joomla API Token tab and click on Reset

If a script having a wrong or revoked Token is called then it will display the following error message: “forbidden”

joomla-api

7.1. How did restrict the rule to our scripts

Let’s suppose that all my scripts filenames start with api.

Then I want to have this htpsswd popup asking only for the files starting with api.

This is why in the snippet of .htaccess below we have <FilesMatch "^api*">

7.2. How did we allow the scripts if launched by the website itself

See the line Require ip 127.0.0.1 185.221.181.208

7.3. More information

For more information about this kind of .htaccess rules, see https://httpd.apache.org/docs/2.4/upgrading.html

8. Using the Joomla API to Import and Update Articles directly from a Google Sheet

GoogleSheets

Being able to POST or PATCH a single Article programmatically with a few lines of code thanks to the API as shown above is great… but the added value in real life is quite limited at this point.

Indeed, having to prepare 1 script to create only 1 article would seem quite cumbersome. But once you can do it for 1, one starts to think “what about having a loop to create multiple articles” :)

When I first spoke about this with Alexandre, I would already have been happy to do it with a CSV file:

But Alexandre went further:

8.1. Prepare Google Sheets

8.1.1. Create Google Sheets

Create a Google Sheets

Now we need to have some sample data with the right structure for our Google Sheets. We don’t want to reinvent the wheel so let’s have a look at the following script:

https://github.com/alexandreelise/j4x-api-examples/blob/master/using-raw-php/smart-add-edit-to-multiple-sites-from-csv-url.php

In particular I see the following line $csvUrl = 'https://docs.google.com/spreadsheets/d/e/2PACX-1vTlM7un4cv3t5oKQ6mymmBDrAnbpFcYLFh6KnHMC6iCE8qUJSNKJ4Vw54r4OjGNNU4DXxOuLWdtKvZ2/pub?output=csv';

Excellent, let’s copy/paste this URL in a browser

https://docs.google.com/spreadsheets/d/e/2PACX-1vTlM7un4cv3t5oKQ6mymmBDrAnbpFcYLFh6KnHMC6iCE8qUJSNKJ4Vw54r4OjGNNU4DXxOuLWdtKvZ2/pub?output=csv

This makes my browser download a CSV file. I could start from this file… but an easier way is to simply display the Google Sheet directly on the browser by dropping the ?output=csv at the end of the URL.

With other words, I open

https://docs.google.com/spreadsheets/d/e/2PACX-1vTlM7un4cv3t5oKQ6mymmBDrAnbpFcYLFh6KnHMC6iCE8qUJSNKJ4Vw54r4OjGNNU4DXxOuLWdtKvZ2/pub

Then

joomla-api

8.1.2. Share Google Sheets

In Google Sheets

In our case, the url is

https://docs.google.com/spreadsheets/d/e/2PACX-1vT5Th1TKSKv3vZuEvTnpK_MPqhItjuyV8ierdbNQa9FV2nYWHV78Cd9IRGbP9BHIXPCmnCrs6Jm5791/pub?gid=0&single=true&output=csv

So if you want to have a preview of that directly on your browser, take the url without what follows /pub, namely

https://docs.google.com/spreadsheets/d/e/2PACX-1vT5Th1TKSKv3vZuEvTnpK_MPqhItjuyV8ierdbNQa9FV2nYWHV78Cd9IRGbP9BHIXPCmnCrs6Jm5791/pub

Screenshots (my computer is obviously is French but it is pretty straigtforward):

joomla-api

joomla-api

joomla-api

8.2. Basic example

8.2.1. Prepare Google Sheets script

Create your Google Sheet script on your website

Customize your Google Sheet script on your website. In your file adapt the following variables

joomla-api

8.2.2. Execute your script

Simply execute your script as already explained above.

Normally this script should not even display anything and just run in the background.

Still the script displays the “raw result”: this is clearly not necessary but allows to visually check that something indeed happened (and once I could see that some rows were not processed and could identify the cause in the Google Sheets for example).

joomla-api

Beware: I have already noted that Google Sheets sometimes take 1 minute or so before actually updating the CSV version (even if other users of the Google Sheet see your changes “live”). So if after executing your script you see no difference, simply wait 1 minute and relaunch it

8.2.3. Go back to or refresh the Articles Manager

When opening the Article Manager, we can indeed check that

joomla-api

And when opening any of them, we can also see that everything is there

Note:

joomla-api

joomla-api

8.2.4. If you want to Update your articles later on

You have probably noted that the first column in the Google Sheet was id, being of course the ID for each article.

Actually we wanted one single script which works both to Create and to Update Articles.

But how to make the script “smart” enough for that?

So at this point, if there is a chance that I execute again this script in the future, I don’t want to re-create the already existing articles.

Therefore, in my Google Sheet

Illustration with my adapted Google Sheet where

joomla-api

And the result in the Articles Manager after running the script again:

joomla-api

Obviously in your Google Sheets you can change the value of any field. Example: language is set on * (meaning all languages since it is the way it is encoded in the database) but you could specify a language like fr-FR

8.2.5. Two rounds to process

The script runs two loops :

8.2.6. Other smart features

Note that the script has also been made as “smart” as possible for different aspects. Examples:

joomla-api

8.3. Advanced example

The following steps are not necessary of course: it only applies if you intend to use the script for example

8.3.1. Create Custom Fields in Joomla

Let’s suppose we create a website for a Rock band. It would be much easier for the musicians to add their new concerts simply in a Google Sheets than to lean how to use a CMS.

Those concerts would then be easily imported and/or updated in the website.

So let’s create a new Category Concerts and assign to it the 3 following Custom Fields :

joomla-api

joomla-api

joomla-api

8.3.2. Adapt the Google Sheet

Since we create a new Category (“Concert” with ID 8), let us change the value of the catidcolumn from 2 to 8 in the Google Sheet.

Let us also create the following columns in the Google Sheet:

Note: when you type date, they should have the following format: 2022-11-23 20:00:00 (and visibly it imports dates in the real TimeZone of your website, as if you were typing them from Joomla’s backend)

Here is an example with some values:

joomla-api

8.3.3. Adapt the script

In your script

joomla-api

        'publish_up',
        'publish_down',
        'featured_up',
        'featured_down',

joomla-api

8.3.4. Go back to or refresh the Articles Manager

Tadam!

Our Categories have indeed been Updated:

joomla-api

Our Custom Fields have indeed been Updated:

joomla-api

Our other native Fields have indeed been Updated:

joomla-api

8.4. Special fields like Images or Custom Fields or Type Subform or URLs

Some special fields entails “multiple values”. So how to handle this?

Suppose you want to import an image or a Custom Field of Type Subform.

As you might be aware, in the database these fields are saved in json format.

So what should we type in our Google Sheet cell ? The easiest way is to save a real Article on the website and see in the database what the corresponding json is.

Note that we don’t need in the Google Sheet to “escape the / character”. With other words we simply type / and not \/

Example for Image (which includes the Intro Image and the Full Image, together with their ALT, Caption etc):

{"image_intro":"images/test.jpg","image_intro_alt":"","float_intro":"","image_intro_caption":"","image_fulltext":"images/test.jpg","image_fulltext_alt":"","float_fulltext":"", "image_fulltext_caption":""}

Example for a Custom Field of Type Subform having in this case 2 Custom Fields (field2 and field7) and 3 values (row0, row1, row2) – note: it also accepts HTML tags but I can’t paste them here in the presentation:

{"row0":{"field2":1410,"field1":"This an article summary 1 for this dessert. Short & Sweet!", “field7”: “What’s up Super Joomlers! Alex here…Proud to be a joomler. Nowadays focusing on Joomla! 4.x Web Services Apis.”}, “row1”:{"field2":1410,"field1":"This an article summary 2 for this dessert. Short & Sweet!", “field7”: “What’s up Super Joomlers! Alex here… Proud to be a joomler. Nowadays focusing on Joomla! 4.x Web Services Apis.”}, “row2”:{"field2":1410,"field1":"This an article summary 3 for this dessert. Short & Sweet! “, ”field7": “What’s up Super Joomlers! Alex here… Proud to be a joomler. Nowadays focusing on Joomla! 4.x Web Services Apis.”}}

Example for the urls Field (those URL A, URL B and URL C which are native in Joomla):

{"urla":"https://alexandree.io","urlatext":"Website","targeta":"","urlb":"https://github.com/alexandreelise", "urlbtext":"Github “,”targetb":"","urlc":"https://twitter.com/mralexandrelise","urlctext":"Twitter", "targetc":""}

Note: my Editor is tweaking my code snippets here. Of course all the " are regular double quotes (and not italic). And the Github icon is also added by the Editor

8.5. Fields with potentially multiple values like Custom Fields of Type List or Checkbox

Suppose you have a Custom Field of Type List or Checkbox. For those CF

In that case

8.6. Fields with potentially multiple values like Custom Field of Type Articles Field by Regular Labs

Articles Fields by Regular Labs is a powerful Type of Custom Field allowing to select other Articles: https://regularlabs.com/articlesfield

A good example : for a Library website you have typically have a Category “Books” and a Category “Authors”. You could then link each Book Article to one or several Author Article(s).

If you have multiple values, I have tested succesfully the following 2 methods (let’s suppose we want to link to Articles having ID 12 and 13):

  1. in the Google Sheet, type 12,13 then Synchronize. After that Edit the Article in the Backend (the 2 values do appear there but are not yet correctly saved in the database) and Save it
  2. in the Google Sheet, type directly the values in JSON-format, ie {"0":"12", "1":"13"} (without spaces), then Synchronise

8.7. Formula in Google Sheet to transform comma-separated values into the desired JSON

[ added on 30.03.2023 ]

Suppose you have multiple values on the Google Sheet in the form of comma-separated values

12,Se,Lan, Dis

In order to import them we need to transform them in JSON-format as explained above:

{"0":"12", "1":"Se", "2":"Lan", "3":"Dis"}