Joomla, Custom Fields and external data ● powerful overrides made easy with Joomlatools Pages

by Marc Dechèvre

Woluweb

Marc

all presentations on https://slides.woluweb.be

1 JoomlaDay USA

This presentation which could alternatively be named
Joomlatools Pages - a beginner’s guide
was made especially for

  • JoomlaDay USA | 24 April 2021 | #jdayusa
    The video content will only be available to paid registrants of https://jdayusa.com
  • JoomlaDay France | 18 June 2021 #jd21fr
    The video will be available for the members of https://www.joomla.fr

2 Introduction

2.1 Demo site

We have built a demo site so that you can see the results of all the actions explained step by step in this presentation

https://pages.joomlacustomfields.org/

The corresponding code is - directly visible on the demo site: https://pages.joomlacustomfields.org/code - can be downloaded directly from the demo website (full backup of the website) - and also on Github (probably an older version): https://github.com/woluweb/Joomlatools-Pages-Demo

2.3 Joomlatools Pages - presentations by the author himself

Johan Janssens, the father of Joomlatools Pages, has already made three general presentations: - JoomlaDay USA | 24 April 2021 | Building a Joomla site with Joomlatools Pages - JoomlaShack Conference | 14 Dec 2020 | The Evolution of Web Publishing from 1990 to 2020 - JoomlaShack Conference | 11 May 2020 | An Interview with Johan Janssens about Joomlatools Pages

2.4 Why this presentation

I share two qualities with Johan Janssens : - we are both Belgians 😀 - we are both Joomla fans 😉

Johan, who is also one of the cofounders of Joomla itself, is of course an excellent developer.


I don’t consider myself as a developer (I don’t code extensions, even if I play a lot with overrides which entails some PHP knowledge).

I am more an integrator and what could be called an power user: I know Joomla by heart and like to use its full capabilities.

And I am definitely a Joomla Custom Fields ‘evangelist’ since it was released with J!3.7 thanks to Allon Moritz aka Laoneo.

Custom Fields are indeed a game changer and has made Joomla much more powerful, especially if you use overrides in order to customise the Layout.


So when I discovered Joomlatools Pages, I immediately saw a great opportunity because - it makes overrides much easier - it integrates the Custom Fields - it allows to filter - it gives new possibilities like fetching data from external sources (json files for example) - and even non-coders or coders who don’t know Joomla can immediately use it


So why this presentation?

Simply because I am always happy to share my enthousiasm a.o. about everything related to Joomla Custom Fields.

And also because I like to learn by doing. So I thought a good way of showing some of the features offered by Joomlatools Pages would be to make a step by step demo.

2.5 Focus on Pages to make overrides including Custom Fields and fetching external data

Joomlatools Pages can already do so many things (plus the fact that features are regularly added/improved) that it would probably take a full day to cover them all…

So in this presentation we will ‘only’ see how to make (blog view and article view) overrides including Custom Fields and fetching external data because even so there is already so much to tell!

As you will see, Pages is so powerful and yet so easy…


In particular, Joomlatools Pages allows to fetch and display collections from - Pages (default) - a collection will be retrieved from within the pages folder. If a model is not specified in the frontmatter, the collection is automatically pages. - Filesystem - a collection will be created from one or more data files in the filesystem. - Database - a collection will be created from any database table. - Web Service - a collection will be created from any URL as long as the URL returns structured data.

3 The initial setup

Before playing with our new tool, let’s prepare Joomlatools Pages and our website

3.1 Install

  • Go to https://github.com/joomlatools/joomlatools-pages/releases
  • Download the latest package
  • Install the ZIP file the usual way :
    • Go to your YOURSITE.COM/administrator > Extensions > Manage > Install
    • Upload the package.

When you see “Installation successfully completed!” you are done
(don’t worry if the interface is a bit different from usual after installation: Pages uses its own installer)

pages_install

For other installation methods, see https://github.com/joomlatools/joomlatools-pages/wiki/Installation

3.2 Nothing new in the administration interface…

Now that Pages is installed (together with the Joomlatools Framework), you will notice… that nothing has changed in the backend 🙂

No new Component, no Configuration, no anything.

Actually this is totally normal…

3.3 Updating Pages

3.3.1 Joomla Update Manager

Note:

  • Pages is now a Joomlatools Framework component, it’s no longer a Joomla component and it’s not “installed” in Joomla. This approach allows Pages to also be run standalone
  • therefore Pages does not use the Joomla Update Manager bc the updater is too basic for the way the framework works (it manages the dependencies, which is not the case of the regular updater)
  • If you want to be notified of new versions of Pages, go to the Github page and simply click on the Watch button

3.3.2 What is the process to update Pages

See this explanation (which also covers When to update Pages):

https://github.com/joomlatools/joomlatools-pages/discussions/760

Don’t forget to also replace your extension(s) if they were updated (which you see in the release’s changelog for example with a fixed - ext:joomla

3.4 Before starting: SEF

Do not run Joomla sites anymore with SEF disabled, this setting should be enabled at all times. The only reason it was allowed not to turn SEF on in Joomla 1.5 was to allow for a solid migration path from 1.0 to 1.5, we are now 15 years later, every Joomla site should have SEF enabled by default.

So if you have not already done so - go to System > Global Configuration > tab ‘Site’ > block ‘SEO Settings’ > set option ‘Search Engine Friendly URLs’ to ‘Yes’

3.5 Before starting: URL Rewriting

Oh btw nobody wants to see that /index.php/ in the url, right?

So if you have not already done so - go to System > Global Configuration > tab ‘Site’ > block ‘SEO Settings’ > set option ‘Use URL Rewriting’ to ‘Yes’ - at the root of your website be sure htacces.txt was renamed into .htaccess

Then the url https://pages.joomlacustomfields.org/index.php/step1 simply becomes https://pages.joomlacustomfields.org/step1

Note: Pages requires ‘SEF’ to be enabled but it can work without ‘SEF rewrite on’. Then of course you need to keep the /index.php/ in the url and do http://example.com/index.php/[page]

3.6 Before starting: default configuration

Pages is build using a “no configuration required, everything has a default” approach.

This means that you do not require anything in your config.php to get started. Anything you add there would override a default.

3.7 Create the folder

Using your File Manager or your FTP connexion, create the following folder and subfolder

/joomlatools-pages/pages/

3.8 Let’s create some content with Custom Fields

  • create an Article Category with Alias: organizations
  • create 3 articles in that Category with an image (intro or full)
  • create a Custom Field (of type Text for example) with the following Name: socialbrusselsid
  • create a Custom Field (of type URL) with the following Name: youtube
  • assign those Custom Fields to the Article Category ‘organizations’
  • assign the following values to the Custom Field ‘socialbrusselsid’: 13817, 3322, 10600 (for example)
  • assign the following values to the Custom Field ‘youtube’: https://www.youtube.com/watch?v=g21IiNrkhXA, https://www.youtube.com/watch?v=Gooof6e6BvQ, https://www.youtube.com/watch?v=LXeaDPmHP18 (or any other YouTube video of course 🙃 I simply chose those ones bc my daughter plays drums with her progressive rock band)

The socialbrusselsid Custom Field will allow us to combine our articles with information fetched from an external website via webservices.

Example: - https://social.brussels/organisation/13817 (‘human’ format) - https://social.brussels/rest/organisation/13817 (json format)

3.9 See the latest version of the source code on the website itself

To make it easy, I have copied the source of each page in the slides of this presentation itself.

But regularly I bring improvements. So if you want to be sure to get the latest version, they are directly visible on the demo website:

https://pages.joomlacustomfields.org/code


Note: how can I display the source code automatically on the website? Txs to Joomlatools Pages itself.

All I have to do is to create a file

code.html.php

with the following content

 ---
@process:
    filters: [highlight]
 ---

<?= source('step6') ?>

where step6 refers to my file step6.html.php which is created later in this presentation.


Note: in this example and on most of the following ones, I have added

@process: filters: [highlight]

in the Frontmatter. This gives automatic code highlighting.

On a production site you would probably remove it, but here I have included it bc I display on purpose the whole array of articles with

<pre><code><?= var_dump(collection()); ?></code></pre>

See https://github.com/joomlatools/joomlatools-pages/discussions/668 for more information.

Note: to change the style from light atom-one-light.min.css to dark atom-one-dark.min.css, simply adapt /components/com_pages/template/filter/highlight.php accordingly

4 Updating from 0.19 to 0.21

JoomlaTools Pages 0.21 was released in June 2021 and came with plenty of improvements and new features.

It can now also run in Standalone mode.

4.1 Release notes

For an overview (and also for more information on a few breaking changes) see https://github.com/joomlatools/joomlatools-pages/discussions/738

4.2 My own checklist for upgrade

  1. if you created Menu Items of type Pages, those don’t exist any more => simply use Menu Items of type System Links > URL instead (one exception to that at the moment: the Home menu item. See https://github.com/joomlatools/joomlatools-pages/discussions/758)
  2. delete the /cache/ directory
  3. create a /templates/ and move the /partials/ directory into it
  4. in all frontmatters where it applies
    4.1. add a @ in order to have respectively @route, @layout, @collection, @form, @process
    4.2. change extend: path : add a slash before the path to get extend: /path
    4.3. change data: items into data_path: items

  1. all routes and paths need to start with / both in frontmatter and in route calls
  2. change <?= import('/partials/embeds/myfile') ?> into <?= partial('/embeds/myfile') ?>
  3. change <?= $item->getRoute() ?> into <?= route($item) ?>

5 A step by step demo

The idea of this step by step demo is to onboard everybody, even completely non technical persons.

5.1 Joomlatools Pages with HTML input

Let’s start with something super easy: a little HTML file.

5.1.1 Create the file

In the folder /joomlatools-pages/pages/ create a file named

step1.html

having the following content
(latest version on https://pages.joomlacustomfields.org/code#step1)

 ---
title: A HTML page
summary: This page is a static HTML page.
 ---
%INCLUDE code/step1.html%

5.1.2 See the result

Go to https://pages.joomlacustomfields.org/step1 and see the result: step1.html is displayed as if it were a regular Joomla article!

So actually, so far we already have a kind of Flat File CMS 🙂

But the nice thing is that you still benefit from - your template with its design (CSS, …), its modules, … - and all the other Joomla features

5.1.3 Metadata

Do you want to customize the Title and the Description of the page ?

This can also be done very easily by adding this at the beginning of your file

 ---
title: A HTML page
summary: This page is a static HTML page.
 ---

See https://github.com/joomlatools/joomlatools-pages/wiki/Metadata for more information

Note: the part between and is called the Frontmatter

5.1.4 Creating a menu item - optional

The following step is not required, however if you want Joomla to show a menu item for your page you can do so by creating one.

In the administrator interface, go to your menu manager and create a new menu item. Strictly speaking the Menu Item Type could be anything: if Pages has a file named after the alias of the menu item, it will take precedence.

So a typical choice would be to create a Menu Item of type ‘System Links > URL’ and set the Link field to /step1

5.1.5 Multilingual websites

Once you have played with Joomlatools Pages it will seem obvious to you: if you have a multilingual website and the url of a page in English is YOURSITE.COM/en/step1 then the corresponding file should be

/joomlatools-pages/pages/en/step1.html

and not

/joomlatools-pages/pages/step1.html

5.2 Joomlatools Pages with MarkDown input

You don’t know what MarkDown is ? - see https://en.wikipedia.org/wiki/Markdown - in a few words, it is a lightweight markup language for creating formatted text using a plain-text editor

5.2.1 Create the file

In the folder /joomlatools-pages/pages/ create a file named

step2.html.md

having the following content
(latest version on https://pages.joomlacustomfields.org/code#step2)

 ---
title: A Markdown page
summary: This page is rendered by the Markdown engine.
 ---
 ## Hello World!

Welcome to my new page in **MarkDown** format.

5.2.2 See the result

Go to https://pages.joomlacustomfields.org/step2 and see the result: step2.html.md is displayed as if it were a regular Joomla article!

All my presentations (including this one) are written in MarkDown format (.md file). This means that I could now very easily display on my website a nice HTML version of those Presentations.

5.3 Joomlatools Pages with PHP input

Let’s start with a simple PHP example

5.3.1 Create the file

In the folder /joomlatools-pages/pages/ create a file named

step3.html.php

having the following content
(latest version on https://pages.joomlacustomfields.org/code#step3)

 ---
title: A PHP page
summary: This page is rendered by the PHP engine.
 ---
%INCLUDE code/step3.html.php%

Note the short syntax
<h2><?= $title ?></h2>
instead of what we usually have in Joomla files
<h2><?php echo $title; ?></h2>

5.3.2 See the result

Go to https://pages.joomlacustomfields.org/step3 and see the result.

5.3.3 Insert markdown directly in your PHP file

With Pages you can also insert directly markdown (or other types of file) in your PHP file with the following syntax:

<ktml:partial format="md">
**Why not** put some *markdown* written directly in our file?
</ktml:partial>

5.4 Joomlatools Pages with PHP input fetching external data

And now a killing feature: how to get data from an external website (in this case in json format) in 1 (one!) line of code.

For this demo, let’s use the open data made from https://social.brussels/ which gives all the details (name, address, phone number, …) of all Social organizations in Brussels.

5.4.1 How does social.brussels work

On that website, if you search and select a given Organization, you will get such an url. Example:
https://social.brussels/organisation/13817

where 13817 is the ID of that organization.

And if you add /rest/ after the domain name, you actually get the same information… in json format
https://social.brussels/rest/organisation/13817

You will note that most of the fields have either a suffix Fr (for French) or Nl for Dutch since our country is multilingual.

Note: if you open the latter url in Firefox then you don’t even need an add-on in the browser to see the json in a structured way and with code highlighting.

5.4.2 Create the file

In the folder /joomlatools-pages/pages/ create a file named

step4.html.php

having the following content
(latest version on https://pages.joomlacustomfields.org/code#step4)

 ---
title: A PHP page fetching external data from json file
summary: This page is rendered by the PHP engine.
 ---
%INCLUDE code/step4.html.php%

5.4.3 See the result

Go to https://pages.joomlacustomfields.org/step4 and see the result.

5.4.4 Caching

And the good news is: external data can be easily cached and you can choose the duration easily.

For more info see: https://github.com/joomlatools/joomlatools-pages/pull/512

By default caching is enabled and the data will not be revalidated.

But thanks to the data([url], [cache]) template function you can customize this.

Example if you want to cache for 24 hours:

<? $data = data([url], '1day'); ?>

In this example we make multiple calls to the same resources.

So the question is: what does happen with the Cache ?

Each time you make a call to data, Pages will check if the resource it has cached is still valid. If it is it will use it from cache, if not it will update. So multiple data() functions to the same url with a diff cache time the lowest time wins.
One caveat here: if you call it with a higher cache time first that call might result in data not being updated, while data lower on the same page with a call with a lower cache time is updated. This would happen only on the first refresh of the page, the second time you render the same page since the data was updated in cache you get the correct results for all calls

5.5 Blog view using table=content from database

It is now time to build a simple Blog view.

Pages allow indeed to query directly any Joomla table

5.5.1 Create the file

In the folder /joomlatools-pages/pages/ create a file named

step5.html.php

having the following content
(latest version on https://pages.joomlacustomfields.org/code#step5)

 ---
@collection:
    model: database?table=content
 ---
%INCLUDE code/step5.html.php%

5.5.2 See the result

Go to https://pages.joomlacustomfields.org/step5 and see the result.

5.5.3 Images options are stored in JSON format in Joomla

If you have already seen Joomla’s native code or made overrides, you know about this: the Article Image’s options are stored in the database in JSON format. So to get the path/name of an image, we must first json_decode it:

<? $article_images  = json_decode($article->images);?>
<img src="<?= $article_images->image_intro; ?>

5.10 Single Article

With Pages we can also directly call an Article by its ID or Alias.

See https://pages.joomlacustomfields.org/single

5.11 Other features

So I am now done with what I really wanted to share with your about Pages.

But Pages ships with many other features.

5.11.1 Decoration

Let us illustrate one of those features, namely the ability to ‘decorate’ (namely add content before/after/…) the component’s view, whatever it is: article, third-party extension, …

5.11.1.1 Create the file

In the folder /joomlatools-pages/pages/ create a file named

decoration.html.php

having the following content
(latest version on https://pages.joomlacustomfields.org/code#decoration)

 ---
@process: 
    decorate: true
 ---
%INCLUDE code/decoration.html.php%
5.11.1.2 See the result

Go to https://pages.joomlacustomfields.org/decoration and see the result.

5.11.1.3 How to name the decoration file

So actually if the URL of the page I want to decorate is MYSITE.COM/blabla then all I need to do to decorate it is to create a file named

blabla.html.php

and customize it.

5.11.3 Filtering

Another feature is Filtering.

This can be done via the Frontmatter, via the url of via the code.

Note: difference between Searching and Filtering: - when you search you start with nothing and you go find something - when you filter you start with everything and you remove items that don’t follow the constraints you provide

5.11.3.1 Create the file

In the folder /joomlatools-pages/pages/ create a file named

filter.html.php

having the following content
(latest version on https://pages.joomlacustomfields.org/code#filter)

 ---
@collection:
    model: ext:joomla.model.articles
    state:
        sort: title
 ---

%INCLUDE code/filter.html.php%
5.11.3.2 See the result

Go to https://pages.joomlacustomfields.org/filter and see the result.

5.11.4 Filtering directly on Custom Fields

Another feature is Filtering directly on Custom Fields.

This can be done directly within your Frontmatter… or even via the url.

5.11.4.1 Create the file

In the folder /joomlatools-pages/pages/ create a file named

filter-cf.html.php

having the following content
(latest version on https://pages.joomlacustomfields.org/code#filter-cf)

 ---
@collection:
    model: ext:joomla.model.articles
    state:
        sort: title
        category: [organizations]
 ---

%INCLUDE code/filter-cf.html.php%
5.11.4.2 See the result

Go to https://pages.joomlacustomfields.org/filter-cf and see the result.

5.11.5 Filter on Tags

You can actually filter on many many things.

Example here with Tags.

For more information, see https://github.com/joomlatools/joomlatools-pages/pull/363 > Filtering a collection for all possibilities

To see some examples see https://pages.joomlacustomfields.org/filter-tags

5.11.6 List of files

Pages is also able to dynamically display a list of files.

See our example on https://pages.joomlacustomfields.org/list

5.11.6.1 Create the file

In the folder /joomlatools-pages/pages/ create a file named

list.html.php

having the following content
(latest version on https://pages.joomlacustomfields.org/code#list)

%INCLUDE code/list.html.php%
5.11.6.2 See the result

Go to https://pages.joomlacustomfields.org/list and see the result.

5.11.7 Table of Content

See https://pages.joomlacustomfields.org/toc

5.11.8 Displaying content from various sources

  • local json: https://pages.joomlacustomfields.org/local
  • external json: https://pages.joomlacustomfields.org/ext-json
  • external json (advanced, with filters): https://pages.joomlacustomfields.org/ext-json-filter
  • external md: https://pages.joomlacustomfields.org/ext-md
  • Google Doc (see hereafter)
  • Google Sheet (see herafter)
  • Airtable (see hereafter)

5.11.9 GDoc

Yet another feature of Joomlatools Pages: it is possible to load any type of structured data file by file by url.

In this example, Pages will extract the content from a published Google Doc and inject it in the page.

For more information see https://github.com/joomlatools/joomlatools-pages/pull/264

5.11.9.1 Create the file

In the folder /joomlatools-pages/pages/ create a file named

gdoc.html.php

having the following content
(latest version on https://pages.joomlacustomfields.org/code#gdoc)

 ---
url: https://docs.google.com/document/d/e/2PACX-1vTK9nkPIsoq6ZJeVm86ns4BX7Q0bOcMHV5jYoaGlsREXUxt22kDW2zt0Zh3wSF9mquuVQKlPFyRw9HK/pub
 ---

%INCLUDE code/gdoc.html.php%
5.11.9.2 See the result

Go to https://pages.joomlacustomfields.org/gdoc and see the result.

5.11.10 GSheet

Go to https://pages.joomlacustomfields.org/gsheet + https://pages.joomlacustomfields.org/gsheet2 and see the result.

You can even filter on a Google Sheet 👍

5.11.11 Airtable

Go to https://pages.joomlacustomfields.org/airtable and see the result based on the corresponding code on https://pages.joomlacustomfields.org/code#airtable

To make this work you first need to add the Airtable Collection Model to JoomlaTools Pages by adding the following code to
/joomlatools-pages/extensions/pages/model/airtable.php

: https://github.com/joomlatools/joomlatools-pages/discussions/623

5.11.12 Sitemap

Pages is able to generate the sitemap automatically.

See https://pages.joomlacustomfields.org/sitemap.xml

5.11.13 Cache

Pages is super-powerful regarding caching.

See https://pages.joomlacustomfields.org/cache

5.11.14 Custom Collection

If you are a coder, you can even create your own collection (similar to ext:joomla.model.articles).

See https://pages.joomlacustomfields.org/custom-collection

5.11.15 Module

Pages is able to inject directly Modules in Joomla in any position.

See https://pages.joomlacustomfields.org/module

6 Recent features

which came along with Pages 0.21 in June 2021

6.1 ACL

Pages also allows to use Joomla’s ACL.

6.1.1 Via the Frontmatter

Simply add the following in the Frontmatter.

You can specify roles (called Access Levels in Joomla) and/or groups :

access:
    roles: [public]
    groups: [public, guest]

6.1.2 Via the code

Since version 0.21 Pages also allows to make it work directly via the code (for example if only a part of the page has restricted access).

And again it is super easy to use:

<h3>Only visible for hasRole or hasGroup</h3>
<? if(user()->hasGroup(['manager', 'administrator'], true)) : ?>
   <p>blabla</p>
<?  endif ?>
<? if(user()->hasRole('special')) : ?>
   <p>blabla</p>
<?  endif ?>

Go to https://pages.joomlacustomfields.org/acl and see the result.

6.2 Transform your site into a webservice

As you might know, Joomla!4 will ship with a Rest API.

But you don’t need to wait: with the next version (0.21) Pages will also allow you to transform your content into json:

Example: - blog version: https://www.joomlatools.com/blog/ - json version: https://www.joomlatools.com/blog.json

The corresponding code: - https://github.com/joomlatools/joomlatools-pages/blob/master/contrib/sites/blog/pages/blog/index.json.php


Pages supports indeed rendering to different machine readable formats: CSV, JSON, RSS, XML. Support for all of those formats is included, and you do not need to write any code. Just create a file with the right format, defined a collection in it and Pages will render the data from the collection in the specific format.

For more informatoin, see https://github.com/joomlatools/joomlatools-pages/discussions/675


See - https://pages.joomlacustomfields.org/webservice - https://pages.joomlacustomfields.org/all

6.3 Standalone

See

  • https://github.com/joomlatools/joomlatools-pages#standalone
  • “Pages is now also standalone. What does it imply (or not)?”
    https://github.com/joomlatools/joomlatools-pages/discussions/750
  • (https://pages.joomlacustomfields.org/standalone)

7 Breaking News 2021 05 21

https://github.com/joomlatools/joomlatools-pages/pull/369 Template (layout and partial) location handling and importing got a major overhaul for 0.21, it’s now possible to distribute templates with extensions

This opens a whole world of flexibility, imagine creating a Pages extension that provides re-usable layouts and partials, together with additional template function and or helpers, all that nicely zipped in a phar for easy distribution.

Don’t know what phar format is? See https://en.wikipedia.org/wiki/PHAR_(file_format)

8 Y O U R next steps

Now time to go and play for yourself!

Curious to know even more? Have a look here to have examples of sites built with Joomlatools Pages: - https://github.com/joomlatools/joomlatools-pages/discussions/categories/show-and-tell - https://github.com/joomlatools/joomlatools-pages/tree/master/contrib/sites/blog
demo at Joomladay USA 2021 - https://github.com/johanjanssens/agrea.ph
complete ource code of a website made by Johan with Joomlatools Pages

9 Conclusion

First of all I would like to thank Johan Janssens for different reasons - he has always been very kind/patient with me asking questions (as a non-developer to a developer) 👍 - he has also improved a series of Pages features following our discussions (in particular regarding Custom Fields which is my favourite area in Joomla) 🎉


Second, I should make a confession - this presentation is already pretty long. My first goal was to cover 1 feature of Pages: Blog View ‘overrides’ and Article View ‘overrides’ - eventually, I ended up making tens of menu items to demo Pages - but actually I am pretty sure I only (dis)covered a fraction of Pages is or will be 😀

10 Get in touch

https://slides.woluweb.be

Any suggestion about this presentation ?
Please feel free to contact me. I’ll be happy to keep improving it:)

Marc Dechèvre | woluweb
+32 474 37 13 12 | +32 2 772 58 69

https://www.woluweb.be/contact