SQL databases tend to be rigid.
If you have worked with them, you would agree that database design though it seems easier, is a lot trickier in practice. SQL databases believe in structure, that is why it’s called structured query language.
On the other side of the horizon, we have the NoSQL databases, also called schema-less databases that encourage flexibility. In schema-less databases, there is no imposed structural restriction, only data to be saved.
Table of Contents
- Why Use JSON
- The Schema
- The CRUD Operations
- A Primer for Web Applications
- Creating the Migrations
- Creating the Models
- Resource Operations
- Curtains
Though every tool has it’s use case, sometimes things call for a hybrid approach.
What if you could structure some parts of your database and leave others to be flexible?
MySQL version 5.7.8 introduces a JSON data type that allows you to accomplish that.
In this tutorial, you are going to learn.
- How to design your database tables using JSON fields.
- The various JSON based functions available in MYSQL to create, read, update, and delete rows.
- How to work with JSON fields using the Eloquent ORM in Laravel.
Why Use JSON
At this moment, you are probably asking yourself why would you want to use JSON when MySQL has been catering to a wide variety of database needs even before it introduced a JSON data type.
The answer lies in the use-cases where you would probably use a make-shift approach.
Let me explain with an example.
Suppose you are building a web application where you have to save a user’s configuration/preferences in the database.
Generally, you can create a separate database table with the id
, user_id
, key
, and value
fields or save it as a formatted string that you can parse at runtime.
However, this works well for a small number of users. If you have about a thousand users and five configuration keys, you are looking at a table with five thousand records that addresses a very small feature of your application.
Or if you are taking the formatted string route, extraneous code that only compounds your server load.
Using a JSON data type field to save a user’s configuration in such a scenario can spare you a database table’s space and bring down the number of records, which were being saved separately, to be the same as the number of users.
And you get the added benefit of not having to write any JSON parsing code, the ORM or the language runtime takes care of it.
The Schema
Before we dive into using all the cool JSON stuff in MySQL, we are going to need a sample database to play with.
So, let’s get our database schema out of the way first.
We are going to consider the use case of an online store that houses multiple brands and a variety of electronics.
Since different electronics have different attributes(compare a Macbook with a Vacuumn Cleaner) that buyers are interested in, typically the Entity–attribute–value model (EAV) pattern is used.
However, since we now have the option to use a JSON data type, we are going to drop EAV.
For a start, our database will be named e_store
and has three tables only named, brands
, categories
, and products
respectively.
Our brands
and categories
tables will be pretty similar, each having an id
and a name
field.
1 | CREATE DATABASE IF NOT EXISTS `e_store` |
The objective of these two tables will be to house the product categories and the brands that provide these products.
While we are at it, let us go ahead and seed some data into these tables to use later.
1 | /* Brands */ |
Next, is the business area of this tutorial.
We are going to create a products
table with the id
, name
, brand_id
, category_id
, and attributes
fields.
1 | CREATE TABLE `e_store`.`products`( |
Our table definition specifies foreign key constraints for the brand_id
and category_id
fields, specifying that they reference the brands
and categories
table respectively. We have also specified that the referenced rows should not be allowed to delete and if updated, the changes should reflect in the references as well.
The attributes
field’s column type has been declared to be JSON which is the native data type now available in MySQL. This allows us to use the various JSON related constructs in MySQL on our attributes
field.
Here is an entity relationship diagram of our created database.
Our database design is not the best in terms of efficiency and accuracy. There is no price column in the
products
table and we could do with putting a product into multiple categories. However, the purpose of this tutorial is not to teach database design but rather how to model objects of different nature in a single table using MySQL’s JSON features.
The CRUD Operations
Let us look at how to create, read, update, and delete data in a JSON field.
Create
Creating a record in the database with a JSON field is pretty simple.
All you need to do is add valid JSON as the field value in your insert statement.
1 | /* Let's sell some televisions */ |
Instead of laying out the JSON object yourself, you can also use the built-in JSON_OBJECT
function.
The JSON_OBJECT
function accepts a list of key/value pairs in the form JSON_OBJECT(key1, value1, key2, value2, ... key(n), value(n))
and returns a JSON object.
1 | /* Let's sell some mobilephones */ |
Notice the JSON_ARRAY
function which returns a JSON array when passed a set of values.
If you specify a single key multiple times, only the first key/value pair will be retained. This is called normalizing the JSON in MySQL’s terms. Also, as part of normalization, the object keys are sorted and the extra white-space between key/value pairs is removed.
Another function that we can use to create JSON objects is the JSON_MERGE
function.
The JSON_MERGE
function takes multiple JSON objects and produces a single, aggregate object.
1 | /* Let's sell some cameras */ |
There is a lot happening in these insert statements and it can get a bit confusing. However, it is pretty simple.
We are only passing objects to the JSON_MERGE
function. Some of them have been constructed using the JSON_OBJECT
function we saw previously whereas others have been passed as valid JSON strings.
In case of the JSON_MERGE
function, if a key is repeated multiple times, it’s value is retained as an array in the output.
A proof of concept is in order I suppose.
1 | /* output: {"network": ["GSM", "CDMA", "HSPA", "EVDO"]} */ |
We can confirm all our queries were run successfully using the JSON_TYPE
function which gives us the field value type.
1 | /* output: OBJECT */ |
Read
Right, we have a few products in our database to work with.
For typical MySQL values that are not of type JSON, a where clause is pretty straight-forward. Just specify the column, an operator, and the values you need to work with.
Heuristically, when working with JSON columns, this does not work.
1 | /* It's not that simple */ |
When you wish to narrow down rows using a JSON field, you should be familiar with the concept of a path expression.
The most simplest definition of a path expression(think JQuery selectors) is it’s used to specify which parts of the JSON document to work with.
The second piece of the puzzle is the JSON_EXTRACT
function which accepts a path expression to navigate through JSON.
Let us say we are interested in the range of televisions that have atleast a single USB and HDMI port.
1 | SELECT |
The first argument to the JSON_EXTRACT
function is the JSON to apply the path expression to which is the attributes
column. The $
symbol tokenizes the object to work with. The $.ports.usb
and $.ports.hdmi
path expressions translate to “take the usb key under ports” and “take the hdmi key under ports” respectively.
Once we have extracted the keys we are interested in, it is pretty simple to use the MySQL operators such as >
on them.
Also, the JSON_EXTRACT
function has the alias ->
that you can use to make your queries more readable.
Revising our previous query.
1 | SELECT |
Update
In order to update JSON values, we are going to use the JSON_INSERT
, JSON_REPLACE
, and JSON_SET
functions. These functions also require a path expression to specify which parts of the JSON object to modify.
The output of these functions is a valid JSON object with the changes applied.
Let us modify all mobilephones to have a chipset property as well.
1 | UPDATE `e_store`.`products` |
The $.chipset
path expression identifies the position of the chipset
property to be at the root of the object.
Let us update the chipset
property to be more descriptive using the JSON_REPLACE
function.
1 | UPDATE `e_store`.`products` |
Easy peasy!
Lastly, we have the JSON_SET
function which we will use to specify our televisions are pretty colorful.
1 | UPDATE `e_store`.`products` |
All of these functions seem identical but there is a difference in the way they behave.
The JSON_INSERT
function will only add the property to the object if it does not exists already.
The JSON_REPLACE
function substitutes the property only if it is found.
The JSON_SET
function will add the property if it is not found else replace it.
Delete
There are two parts to deleting that we will look at.
The first is to delete a certain key/value from your JSON columns whereas the second is to delete rows using a JSON column.
Let us say we are no longer providing the mount_type
information for cameras and wish to remove it for all cameras.
We will do it using the JSON_REMOVE
function which returns the updated JSON after removing the specified key based on the path expression.
1 | UPDATE `e_store`.`products` |
For the second case, we also do not provide mobilephones anymore that have the Jellybean version of the Android OS.
1 | DELETE FROM `e_store`.`products` |
As stated previously, working with a specific attribute requires the use of the JSON_EXTRACT
function so in order to apply the LIKE
operator, we have first extracted the os
property of mobilephones(with the help of category_id
) and deleted all records that contain the string Jellybean
.
A Primer for Web Applications
The old days of directly working with a database are way behind us.
These days, frameworks insulate developers from lower-level operations and it almost feels alien for a framework fanatic not to be able to translate his/her database knowledge into an object relational mapper.
For the purpose of not leaving such developers heartbroken and wondering about their existence and purpose in the universe, we are going to look at how to go about the business of JSON columns in the Laravel framework.
We will only be focusing on the parts that overlap with our subject matter which deals with JSON columns. An in-depth tutorial on the Laravel framework is beyond the scope of this piece.
Creating the Migrations
Make sure to configure your Laravel application to use a MySQL database.
We are going to create three migrations for brands
, categories
, and products
respectively.
1 | $ php artisan make:migration create_brands |
The create_brands
and create_categories
migrations are pretty similar and and a regulation for Laravel developers.
1 | /* database/migrations/create_brands.php */ |
The create_products
migration will also have the directives for indexes and foreign keys.
1 | /* database/migrations/create_products */ |
Pay attention to the $table->json('attributes');
statement in the migration.
Just like creating any other table field using the appropriate data type named method, we have created a JSON column using the json
method with the name attributes
.
Also, this only works for database engines that support the JSON data type.
Engines, such as older versions of MySQL will not be able to carry out these migrations.
Creating the Models
Other than associations, there is not much needed to set up our models so let’s run through them quickly.
1 | /* app/Brand.php */ |
Again, our Product
model needs a special mention.
The $casts
array which has the key attributes
set to array
makes sure whenever a product is fetched from the database, it’s attributes
JSON is converted to an associated array.
We will see later in the tutorial how this facilitates us to update records from our controller actions.
Resource Operations
Creating a Product
Speaking of the admin panel, the parameters to create a product maybe coming in through different routes since we have a number of product categories. You may also have different views to create, edit, show, and delete a product.
For example, a form to add a camera requires different input fields than a form to add a mobilephone so they warrant separate views.
Moreover, once you have the user input data, you will most probabaly run it through a request validator, separate for the camera, and the mobilephone each.
The final step would be to create the product through Eloquent.
We will be focusing on the camera resource for the rest of this tutorial. Other products can be addressed using the code produced in a similar manner.
Assuming we are saving a camera and the form fields are named as the respective camera attributes, here is the controller action.
1 | // creates product in database |
Fetching Products
Recall the $casts
array we declared earlier in the Product
model. It will help us read and edit a product by treating attributes as an associative array.
1 | // fetches a single product |
Your view would use the $camera
variable in the following manner.
1 | <table> |
Editing a Product
As shown in the previous section, you can easily fetch a product and pass it to the view, which in this case would be the edit view.
You can use the product variable to pre-populate form fields on the edit page.
Updating the product based on the user input will be pretty similar to the store
action we saw earlier, only that instead of creating a new product, you will fetch it first from the database before updating it.
Searching Based on JSON Attributes
The last piece of the puzzle that remains to discuss is querying JSON columns using the Eloquent ORM.
If you have a search page that allows cameras to be searched based on their specifications provided by the user, you can do so with the following code.
1 | // searches cameras by user provided specifications |
The retrived records will now be available to the product.camera.search
view as a $cameras
collection.
Deleting a Product
Using a non-JSON column attribute, you can delete products by specifying a where clause and then calling the delete
method.
For example, in case of an ID.
1 | \App\Product::where('id', $id)->delete(); |
For JSON columns, specify a where clause using a single or multiple attributes and then call the delete
method.
1 | // deletes all cameras with the sensor_type attribute as CMOS |
Curtains
We have barely scratched the surface when it comes to using JSON columns in MySQL.
Whenever you need to save data as key/value pairs in a separate table or work with flexible attributes for an entity, you should consider using a JSON data type field instead as it can heavily contribute to compressing your database design.
If you are interested in diving deeper, the MySQL documentation is a great resource to explore JSON concepts futher.
来源: