As a software engineer, I'm intimately familiar with the thrill of solving complex problems, the satisfaction of optimizing code, and the endless pursuit of knowledge in a rapidly evolving field.

Yet.. there's another realm that has captured my attention and countless hours of my time (and I'm sure many are guilty of this as well), offering a unique blend of intellectual stimulation and addictive exploration: Wikipedia.

I don't even binge on it that often, but when I do.. it always goes on into a rabbithole I never thought I'd encounter.

The Graph of Human Knowledge

At its core, Wikipedia represents something far more fascinating than just another informational website. For those of us who work with complex systems and data structures, it embodies a vast, interconnected web of human knowledge, constantly evolving and expanding.

The structure of Wikipedia bears a striking resemblance to the graph data structures we often work with in software engineering. Each article serves as a node, with hyperlinks acting as edges connecting related concepts. This network of information creates a playground for curious minds, especially those trained to navigate and analyze complex systems.

Consider the parallels between Wikipedia's structure and common data structures we use in software development:

1. Directed Graph: Wikipedia's link structure forms a directed graph, where articles (nodes) are connected by hyperlinks (directed edges).
2. Tree Structure: Categories and subcategories in Wikipedia form a hierarchical tree structure, similar to file systems or organizational charts.
3. Hash Table: The search functionality in Wikipedia can be thought of as a hash table, where article titles are keys that quickly lead to the corresponding content.

Understanding these structural similarities can help explain why software engineers often find Wikipedia so engaging – it appeals to our innate sense of structure and organization while providing an endless sea of content to explore.

The (almost) interesting history of Port Numbers

Port numbers are the numerical identifiers that allow computers to communicate over the internet. They are assigned to different applications and protocols, such as web browsing, email, file transfer, and so on. But how did these port numbers come to be? Who decided which port number belongs to which service?

WirekatMarkus

The Accidental Time Traveler

One of the most captivating aspects of Wikipedia is its ability to transport us through time and space, often in unexpected ways. This phenomenon, affectionately known as "Wiki rabbit holes," is a testament to the power of interconnected information and our own curiosity.

Let me share a personal anecdote that illustrates this point:

Last week, I set out to quickly look up the time complexity of quicksort for a university assignment. Two hours later, I found myself deep in an article about the Antikythera mechanism, an ancient Greek analog computer.

Here's how that journey unfolded:Quicksort article → Mention of Tony Hoare, its inventorTony Hoare's page → His work on communicating sequential processesCSP article → Its influence on concurrent programming languagesConcurrent programming → Real-time computing applicationsReal-time computing → Its use in astronomy and celestial mechanicsCelestial mechanics → Ancient methods of predicting celestial eventsAncient astronomy → The Antikythera mechanism

We follow threads of interest, make unexpected connections, and continuously expand our understanding of the world around us.

It's very similar to how many use the software, Obsidian, to create connections and links to related pages in their markdown pages.

The Joy of Unexpected Connections

As humans, we often find joy in discovering unexpected connections between seemingly unrelated concepts. Wikipedia excels at facilitating these serendipitous discoveries. The ability to jump from one topic to another, following a trail of hyperlinks, mimics the way our minds make associations and draw parallels between different ideas.

Here's another example of how a simple query about a programming concept led to a fascinating journey through history and science:

1. Started by looking up information about recursion in programming.
2. Came across the concept of "turtles all the way down" in the article's "In popular culture" section.
3. This led to reading about the "unmoved mover" in philosophy and cosmology.
4. From there, explored articles on the Big Bang theory and cosmic inflation.
5. Inflation theory led to reading about quantum fluctuations and the uncertainty principle.
6. This circled back to computer science with quantum computing and its potential impact on cryptography.
7. Ended up reading about post-quantum cryptography and the race to develop new encryption methods.

This intellectual journey, spanning computer science, philosophy, cosmology, and quantum physics, exemplifies the kind of interdisciplinary exploration that Wikipedia facilitates. It's this ability to connect disparate fields of knowledge that makes Wikipedia particularly addictive for those with curious, analytical minds.

The 10 Most Difficult Programming Concepts to Grasp

Programming is a skill that requires a lot of logical thinking, creativity, and patience. It can also be very rewarding and fun, especially when you can create something useful or beautiful with code. However, programming is not always easy. There are some concepts that are notoriously difficult to understand, even

WirekatMarkus

The Dopamine Hit of Learning

There's a neurochemical aspect to Wikipedia addiction that resonates particularly well with the software engineering mindset. Every new piece of information, every "aha!" moment when we connect two previously unrelated concepts, triggers a small dopamine release in our brains. This is the same neurotransmitter associated with the satisfaction we feel when solving a tricky bug or optimizing a piece of code.

Dr. Judy Willis, a neurologist and former teacher, explains that the dopamine release associated with learning new information can be just as rewarding as winning money or eating chocolate. For those of us who thrive on problem-solving and continuous learning, Wikipedia provides an endless stream of these small, rewarding moments.

Consider the following cycle that often occurs during a Wikipedia session:

1. Encounter an unfamiliar term or concept
2. Feel a sense of curiosity and mild cognitive dissonance
3. Click the link to learn more
4. Experience a small sense of accomplishment and pleasure upon understanding
5. Notice new, related concepts in the article
6. Repeat from step 1

This cycle, driven by our brain's reward system, can keep us engaged for hours, constantly seeking the next bit of knowledge and the accompanying dopamine hit.

The Flow State and Wikipedia

Another aspect of Wikipedia's addictive nature is its ability to induce a flow state, a concept popularized by psychologist Mihály Csíkszentmihályi. Flow is a mental state of complete absorption in an activity, often characterized by a sense of energized focus, full involvement, and enjoyment in the process of the activity.

As software engineers, we're familiar with entering flow states while coding or solving complex problems. Wikipedia browsing can induce a similar state, where time seems to fly by as we immerse ourselves in a sea of interconnected knowledge.

The conditions that facilitate flow states align well with the Wikipedia experience:

1. Clear goals (finding information on a specific topic)
2. Immediate feedback (instantly accessing new information)
3. A balance between challenge and skill (learning new concepts without being overwhelmed)
4. A sense of control (choosing which links to follow)
5. Deep concentration (focusing on absorbing new information)

Understanding these psychological and neurological factors can help explain why Wikipedia browsing can be so captivating and, at times, difficult to stop.

Falling Down the Rabbit Hole: Interesting Articles

To illustrate the addictive nature of Wikipedia browsing, let's look at a few examples of articles that are particularly captivating for those with a technical mindset. Each of these can serve as a starting point for hours of fascinating exploration:

1. The Halting Problem This article delves into one of the fundamental concepts of computer science, discussing the limits of computation and the philosophical implications of unsolvable problems. It's a great starting point for a journey into theoretical computer science and the foundations of our field. Key points of interest:
   • Alan Turing's proof of the problem's undecidability
   • Connections to Gödel's incompleteness theorems
   • Implications for software verification and bug detection
   • Related decidable and undecidable problems in computer science
2. Brainfuck An esoteric programming language that, despite its crude name, offers fascinating insights into the nature of Turing completeness and the minimum instructions required for computation. Reading about this language often leads to explorations of other esoteric languages and the limits of programming paradigms. Interesting aspects:
   • The language's minimalist design (only 8 commands)
   • Examples of Brainfuck programs and their equivalents in other languages
   • Discussion of Turing completeness and computational universality
   • Links to other esoteric programming languages like Piet and Whitespace
3. RSA (cryptosystem) This article not only explains one of the most widely used public-key cryptosystems but also touches on number theory, the history of cryptography, and the ongoing challenges in information security. It's a perfect blend of mathematics, computer science, and real-world applications. Key topics:
   • The mathematical foundations of RSA (modular arithmetic, prime factorization)
   • The history of public-key cryptography
   • Practical implementations and optimizations of RSA
   • Attacks on RSA and their mitigations
   • The impact of quantum computing on RSA's security
4. Byzantine fault tolerance A concept crucial to distributed systems and blockchain technology, this article opens up a world of exploration into consensus algorithms, fault-tolerant design, and the challenges of building reliable systems in unreliable environments. Interesting tangents:
   • The Byzantine Generals' Problem and its variations
   • Practical Byzantine Fault Tolerance (pBFT) algorithm
   • Applications in blockchain and cryptocurrency systems
   • Connections to game theory and mechanism design
   • Real-world examples of Byzantine faults in computer systems
5. Quantum supremacy This topic bridges computer science, physics, and the future of computation. It's a gateway to articles on quantum mechanics, the limits of classical computing, and the potential revolutionary impact of quantum computers on fields like cryptography and optimization. Fascinating aspects:
   • The race between tech giants to achieve quantum supremacy
   • Debate over the practical significance of quantum supremacy demonstrations
   • Potential applications of quantum computing in various fields
   • Challenges in building and scaling quantum computers
   • The impact on current encryption methods and the development of post-quantum cryptography

These articles serve as excellent entry points to vast areas of knowledge. Each link within these articles can lead to new discoveries and connections, fueling the addictive cycle of learning and exploration that makes Wikipedia so captivating.

When Code Goes Catastrophically Wrong

A tiny bug can sometimes lead to colossal consequences. After listening to too many technical podcasts in my spare time.. Let’s dive into three infamous cases where a few lines of code caused chaos, destruction, and in one case, an interplanetary facepalm. We’ll also see how they could have been

WirekatMarkus

The Pitfalls of Wikipedia Addiction

While the vast sea of knowledge on Wikipedia can be incredibly enriching, it's also important to acknowledge its potential downsides, especially for those of us prone to falling into "just one more article" syndrome.

1. Time Management Challenges

Wikipedia's addictive trait can lead us to spend hours chasing down every last detail on a Wikipedia topic, long past the point of diminishing returns. This can result in:

• Neglecting important tasks or deadlines
• Disrupted sleep patterns due to late-night reading sessions
• Reduced productivity in our primary work

2. Information Overload

The sheer volume of information available can sometimes be overwhelming, leading to:

• Difficulty in retaining and processing all the consumed information
• Cognitive fatigue from trying to absorb too much diverse information in a short time
• Potential confusion when dealing with conflicting information or edits in progress

3. Shallow Learning

While Wikipedia is excellent for getting an overview of a topic, relying on it exclusively can sometimes lead to:

• A false sense of expertise in topics we've only briefly skimmed
• Missing out on the depth that specialized textbooks or courses can provide
• Difficulty in critically analyzing information due to the rapid consumption of facts

4. Distraction from Practical Application

For software engineers, there's a risk that excessive Wikipedia browsing can become a form of procrastination, preventing us from:

• Applying the knowledge we've gained to real-world problems
• Engaging in hands-on coding practice
• Developing our own original ideas and solutions

Conclusion

In a world that can sometimes feel divided, Wikipedia stands as a shining example of global collaboration. People from all walks of life, across countless cultures and languages, come together to build this vast repository of information. It's a reminder that, at our core, humans are naturally curious beings who delight in learning and sharing what we know.

A tiny bug can sometimes lead to colossal consequences. After listening to too many technical podcasts in my spare time.. Let's dive into three infamous cases where a few lines of code caused chaos, destruction, and in one case, an interplanetary facepalm.

We'll also see how they could have been solved, in a very simplified way!

1. The Rocket That Went Boom: Ariane 5 Flight 501

Imagine spending a decade and $7 billion to build a rocket, only to watch it self-destruct 37 seconds after launch. That's exactly what happened to the European Space Agency's Ariane 5 rocket in 1996. The culprit? A little thing called integer overflow.

The rocket's inertial reference system tried to stuff a 64-bit floating point number into a 16-bit integer. Oops! This caused the guidance system to go haywire, resulting in an un-commanded change of trajectory. In simpler terms, the rocket thought it was drunk and decided to break dance instead of flying straight.

How could this have been prevented? By using proper exception handling and range checking. Here's a simplified example of what they should have done:

def convert_velocity(velocity_64bit):
    max_16bit = 32767  # Maximum value for a 16-bit integer
    
    try:
        velocity_16bit = int(velocity_64bit)
        if velocity_16bit > max_16bit:
            raise ValueError("Velocity out of range for 16-bit conversion")
        return velocity_16bit
    except ValueError as e:
        log_error(f"Conversion error: {e}")
        return fallback_safe_velocity()

The Ariane 5 used Ada, a language designed for embedded and real-time systems. Here's an Ada-like representation of the problem and a potential fix:

-- Original problematic code (simplified)
procedure Convert_Velocity is
   Velocity_64 : Long_Float;
   Velocity_16 : Integer_16;  -- 16-bit integer type
begin
   Velocity_64 := Get_Velocity;  -- This function returns a 64-bit float
   Velocity_16 := Integer_16(Velocity_64);  -- This conversion could cause overflow
end Convert_Velocity;

-- Improved version with error handling
procedure Convert_Velocity (Success : out Boolean) is
   Velocity_64 : Long_Float;
   Velocity_16 : Integer_16;
   Max_16 : constant := 32767;  -- Maximum value for 16-bit signed integer
begin
   Success := False;
   Velocity_64 := Get_Velocity;
   
   if Velocity_64 > Long_Float(Max_16) or Velocity_64 < Long_Float(-Max_16) then
      -- Log error and use fallback value
      Log_Error("Velocity out of range for 16-bit conversion");
      Velocity_16 := Fallback_Safe_Velocity;
   else
      Velocity_16 := Integer_16(Velocity_64);
      Success := True;
   end if;
exception
   when others =>
      Log_Error("Unexpected error in velocity conversion");
      Velocity_16 := Fallback_Safe_Velocity;
end Convert_Velocity;

Lesson learned: Always validate your inputs and handle potential overflows!

2. The $125 Million Typo: NASA's Mars Climate Orbiter

In 1999, NASA's Mars Climate Orbiter got a little too close and personal with the Red Planet, disintegrating in its atmosphere. The reason? A simple unit conversion error. One team used metric units (newtons) while another used imperial units (pound-force). It's like trying to bake a cake with a recipe that switches between cups and milliliters without telling you.

This metric mixup caused the orbiter to approach Mars at the wrong angle, turning a scientific mission into a very expensive shooting star.

How could this planetary faux pas have been prevented? By using a clear unit standard and implementing rigorous code reviews. Here's a simple example of how they could have handled unit conversions:

class Force:
    def __init__(self, value, unit):
        self.value = value
        self.unit = unit.lower()

    def to_newtons(self):
        if self.unit == 'n' or self.unit == 'newtons':
            return self.value
        elif self.unit == 'lbf' or self.unit == 'pound-force':
            return self.value * 4.448222  # Conversion factor
        else:
            raise ValueError(f"Unsupported unit: {self.unit}")

# Usage
thrust = Force(100, 'lbf')
thrust_in_newtons = thrust.to_newtons()
print(f"Thrust: {thrust_in_newtons} N")

The Mars Climate Orbiter software was written in C. Here's a C representation of the unit conversion issue and a potential solution:

#include <stdio.h>

// Original problematic code (simplified)
double apply_thrust(double force) {
    // Assume force is in pound-force, but the function expects newtons
    return calculate_trajectory(force);
}

// Improved version with explicit unit conversion
#define LBF_TO_NEWTON 4.448222

typedef enum {
    NEWTON,
    POUND_FORCE
} ForceUnit;

typedef struct {
    double value;
    ForceUnit unit;
} Force;

double force_to_newtons(Force force) {
    if (force.unit == NEWTON) {
        return force.value;
    } else if (force.unit == POUND_FORCE) {
        return force.value * LBF_TO_NEWTON;
    } else {
        fprintf(stderr, "Error: Unknown force unit\n");
        return 0.0;
    }
}

double apply_thrust(Force force) {
    double force_in_newtons = force_to_newtons(force);
    return calculate_trajectory(force_in_newtons);
}

int main() {
    Force thrust = {100.0, POUND_FORCE};
    double trajectory = apply_thrust(thrust);
    printf("Trajectory calculated with thrust: %f N\n", force_to_newtons(thrust));
    return 0;
}

Lesson learned: Standardize your units and always double-check your conversions!

3. The Bug That Fried Patients: Therac-25 Radiation Therapy Machine

In the mid-1980s, a radiation therapy machine called the Therac-25 began to malfunction in the worst way possible. Due to a race condition in the code, the machine would occasionally give patients radiation doses that were hundreds of times higher than intended. This tragic bug led to several deaths and numerous injuries.

The problem stemmed from a perfect storm of software issues: poor synchronization between concurrent processes, inadequate error checking, and a false sense of security from previous, safer models.

How could this have been prevented? By implementing thorough error checking, fail-safe mechanisms, and extensive testing. Previous models also had hardware interlocks to prevent such faults, but the Therac-25 had removed them, depending instead on software checks for safety.

Here's a simplified example of how they could have added some basic safety checks:

class RadiationMachine:
    MAX_SAFE_DOSE = 5  # Grays

    def __init__(self):
        self.current_dose = 0
        self.is_beam_on = False

    def set_dose(self, dose):
        if dose > self.MAX_SAFE_DOSE:
            raise ValueError(f"Dose exceeds safe limit of {self.MAX_SAFE_DOSE} Gy")
        self.current_dose = dose

    def activate_beam(self):
        if not self.is_beam_on and self.current_dose <= self.MAX_SAFE_DOSE:
            self.is_beam_on = True
            print("Beam activated")
        else:
            print("ERROR: Cannot activate beam")

    def emergency_shutdown(self):
        self.is_beam_on = False
        self.current_dose = 0
        print("EMERGENCY SHUTDOWN ACTIVATED")

# Usage
machine = RadiationMachine()
try:
    machine.set_dose(3)
    machine.activate_beam()
except ValueError as e:
    print(f"Error: {e}")
    machine.emergency_shutdown()

The Therac-25 used a custom assembly language. Here's a simplified pseudocode representation in a mix of assembly-like syntax and higher-level constructs to illustrate the issues and a safer approach:

; Simplified pseudocode for Therac-25 (not actual assembly)

; Original problematic code (conceptual)
SetDose:
    MOV dose, A         ; Set dose from input
    JMP ActivateBeam    ; Immediately activate beam

ActivateBeam:
    MOV 1, beam_status  ; Turn beam on
    RET

; Improved version with safety checks
SetDose:
    MOV dose, A
    CMP A, MAX_SAFE_DOSE
    JG Error            ; Jump if dose > MAX_SAFE_DOSE
    MOV A, current_dose
    RET

ActivateBeam:
    CMP beam_status, 1
    JE Error            ; Jump if beam already on
    CMP current_dose, MAX_SAFE_DOSE
    JG Error            ; Jump if dose too high
    MOV 1, beam_status
    CALL InitiateSafetyChecks
    RET

InitiateSafetyChecks:
    ; (Additional safety check procedures)
    RET

Error:
    CALL EmergencyShutdown
    RET

EmergencyShutdown:
    MOV 0, beam_status
    MOV 0, current_dose
    ; (Additional shutdown procedures)
    RET

; Main control loop
Main:
    CALL GetUserInput
    CALL SetDose
    CALL ActivateBeam
    JMP Main

Lesson learned: When lives are at stake, implement multiple layers of safety checks and fail-safe mechanisms!

Conclusion

In conclusion, these cautionary tales remind us that in the world of coding, a small oversight can lead to massive consequences. Always validate your inputs, standardize your units, implement thorough error checking, and never, ever skip those code reviews.

Nowadays it's quite standard and even expected to write tests, so make sure to create meaninful ones - no excuses!

Wikipedia is an irresistable rabbithole

As a software engineer, I’m intimately familiar with the thrill of solving complex problems, the satisfaction of optimizing code, and the endless pursuit of knowledge in a rapidly evolving field. Yet.. there’s another realm that has captured my attention and countless hours of my time (and I’m sure many are

WirekatMarkus

In the world of database management, much of your data is most likely never actually deleted. The concept of deletion has evolved over time.

Enter soft deletion - a well-known technique that's changed how we think about data lifecycle management. This post explores the ins and outs of soft deletion, its benefits, and how to implement it effectively.

What is Soft Deletion?

Imagine you're managing a bustling digital library. Instead of permanently removing books from your shelves (hard deletion), soft deletion is like moving them to a special reserved section. They're out of the main circulation, but still accessible if needed.

In technical terms, soft deletion involves marking records as deleted in the database, rather than permanently removing them. This is typically achieved by adding a boolean flag or better, a timestamp to indicate when a record was "deleted."

The Advantages of Going Soft (on Deletion)

1. Data Recovery: Accidentally deleted important information? With soft deletion, recovery is just a database query away.
2. Audit Trails: Maintain a complete history of data changes, crucial for compliance and forensic analysis.
3. Performance Optimization: Avoid the potential performance hit of cascading deletes in large, complex datasets.
4. Trend Analysis: Analyze historical trends including "deleted" data, providing a more complete picture of your data over time.
5. Enhanced User Experience: Allow users to recover their own deleted data, adding a layer of forgiveness to your application.

Optimizing SQL Queries - PostgreSQL best practices

Optimizing SQL queries is essential for improving the performance of a database system. In PostgreSQL, like in any other relational database, the way data is structured and indexed plays a crucial role in query optimization. This article will outline the best practices for structuring data, indexing, and query optimization. So

WirekatMarkus

How Soft Deletion Works

1. Deletion Flag: Add a boolean column (e.g., is_deleted) or a timestamp column (e.g., deleted_at) to your tables.
2. Modified Queries: Instead of using DELETE statements, update the deletion flag to mark records as deleted.
3. Filtered Results: Adjust SELECT queries to exclude "deleted" records by default.
4. Recovery Mechanism: Implement a process to "undelete" by resetting the deletion flag.

Handling Related Data: Cascading Soft Deletes

When dealing with related tables, it's important to ensure that soft deletes cascade properly. There are two main approaches:

1. Database Triggers: Create triggers that automatically propagate the soft delete to related records.
2. Application Logic: Handle cascading deletes in your application code, offering more flexibility but requiring more development effort.

For some tables, cascading may not even be required - but that again, depends on your application. If you find that your data is fine without cascades, then that's fine.

Implementing Soft Deletion in PostgreSQL

Let's look at a practical example of implementing soft deletion in PostgreSQL:

Create tables with a soft delete column:

CREATE TABLE users (
    id SERIAL PRIMARY KEY,
    name VARCHAR(100),
    email VARCHAR(100),
    deleted_at TIMESTAMP
);

CREATE TABLE posts (
    id SERIAL PRIMARY KEY,
    user_id INTEGER REFERENCES users(id),
    title VARCHAR(200),
    content TEXT,
    deleted_at TIMESTAMP
);

Use soft deletion in your queries:

-- Soft delete a user
UPDATE users SET deleted_at = NOW() WHERE id = 1;

-- Select only non-deleted users
SELECT * FROM users WHERE deleted_at IS NULL;

-- Restore a soft-deleted user
UPDATE users SET deleted_at = NULL WHERE id = 1;

For most applications, these changed to your tables is probably enough. You can of course make it more complex and make related data automatically cascade (soft delete) as well.

Below you can see how that can be done, using triggers and functions!

Create a function for soft deletion:

CREATE OR REPLACE FUNCTION soft_delete()
RETURNS TRIGGER AS $$
BEGIN
    NEW.deleted_at = NOW();
    RETURN NEW;
END;
$$ LANGUAGE plpgsql;

This little piece of code is the heart of our soft deletion system. When you create it, you're essentially telling PostgreSQL, "Hey, I've got a special way of handling deletes." Instead of actually removing a row, this function simply marks it as deleted by setting its deleted_at field to the current time. It's like putting a timestamp on a piece of paper to show when it was "thrown away", but you're really just putting it in a special drawer instead of the trash can.

Set up triggers for soft deletion:

CREATE TRIGGER soft_delete_user
BEFORE UPDATE ON users
FOR EACH ROW
WHEN (NEW.deleted_at IS NOT NULL AND OLD.deleted_at IS NULL)
EXECUTE FUNCTION soft_delete();

CREATE TRIGGER soft_delete_post
BEFORE UPDATE ON posts
FOR EACH ROW
WHEN (NEW.deleted_at IS NOT NULL AND OLD.deleted_at IS NULL)
EXECUTE FUNCTION soft_delete();

Now, onto the soft delete triggers. These are like vigilant gatekeepers for your tables. You set them up on your users and posts tables, and they stand guard, watching for any attempts to update the deleted_at field. When they see an update that's trying to change deleted_at from null to a timestamp, they spring into action. They call our soft delete function, which does the actual work of setting the timestamp. It's like having a librarian who, instead of letting you remove a book from the library, simply marks it as "checked out indefinitely" whenever you try to take it.

Implement cascading soft deletes:

CREATE OR REPLACE FUNCTION cascade_soft_delete_user()
RETURNS TRIGGER AS $$
BEGIN
    UPDATE posts SET deleted_at = NEW.deleted_at
    WHERE user_id = NEW.id AND deleted_at IS NULL;
    RETURN NEW;
END;
$$ LANGUAGE plpgsql;

CREATE TRIGGER cascade_soft_delete_user
AFTER UPDATE ON users
FOR EACH ROW
WHEN (NEW.deleted_at IS NOT NULL AND OLD.deleted_at IS NULL)
EXECUTE FUNCTION cascade_soft_delete_user();

Finally, we have the cascading soft delete function and trigger. This is where things get a bit more sophisticated. Imagine you have a user who has written several posts. If you delete the user, you probably want to delete their posts too, ...right? Actually.. keep it if it's a public forum please!

That's what this function does. When a user is soft deleted, this function goes through all the posts associated with that user and marks them as deleted too. The trigger is what kicks off this process. It's like telling the librarian, "Hey, when you mark this author as inactive, also mark all their books as unavailable."

Restoring data, one query at a time

Let's say you messed up and soft deleted a whole table. Or maybe you want to restore some rows using a point in time. Fortunately, your rows have the timestamps already marked!

You can use your SQL knowledge to make sufficient selects and restore the deleted rows.

-- Assume we want to restore users deleted on July 1, 2023 between 2:00 PM and 2:05 PM

-- First, let's see which users were deleted in this time range
SELECT id, name, email, deleted_at
FROM users
WHERE deleted_at BETWEEN '2023-07-01 14:00:00' AND '2023-07-01 14:05:00';

-- Now, let's restore these users by setting their deleted_at to NULL
UPDATE users
SET deleted_at = NULL
WHERE deleted_at BETWEEN '2023-07-01 14:00:00' AND '2023-07-01 14:05:00';

-- Verify the restoration
SELECT id, name, email, deleted_at
FROM users
WHERE deleted_at IS NULL
  AND id IN (
    SELECT id
    FROM users
    WHERE deleted_at BETWEEN '2023-07-01 14:00:00' AND '2023-07-01 14:05:00'
  );

-- If you need to restore associated data (like posts), you might do:
UPDATE posts
SET deleted_at = NULL
WHERE user_id IN (
    SELECT id
    FROM users
    WHERE deleted_at BETWEEN '2023-07-01 14:00:00' AND '2023-07-01 14:05:00'
);

Best Practices for Effective Soft Deletion

Indexing: Add an index to the deleted_at column to maintain query performance.

CREATE INDEX idx_users_deleted_at ON users (deleted_at);
CREATE INDEX idx_posts_deleted_at ON posts (deleted_at);

Partial Indexes: Utilize partial indexes to exclude soft-deleted records from your indexes.

-- Partial Indexes: Exclude soft-deleted records from indexes
CREATE INDEX idx_active_users_email ON users (email) WHERE deleted_at IS NULL;
CREATE INDEX idx_active_posts_title ON posts (title) WHERE deleted_at IS NULL;

Periodic Hard Deletes: Implement a process to permanently delete old soft-deleted records to manage database size. For example, to delete rows that have been deleted years ago.

-- Periodic Hard Deletes: Create a function to permanently delete old soft-deleted records
CREATE OR REPLACE FUNCTION permanently_delete_old_records()
RETURNS void AS $$
BEGIN
    -- Delete users soft-deleted more than 1 year ago
    DELETE FROM users WHERE deleted_at < NOW() - INTERVAL '1 year';
    
    -- Delete posts soft-deleted more than 6 months ago
    DELETE FROM posts WHERE deleted_at < NOW() - INTERVAL '6 months';
END;
$$ LANGUAGE plpgsql;

1. Unique Constraints: Be mindful of unique constraints when using soft deletes, especially if you plan to reuse unique values.

CREATE UNIQUE INDEX idx_active_users_email_unique ON users (email) WHERE deleted_at IS NULL;

1. API Design: Design your API to handle soft deletes transparently, maintaining a consistent experience for consumers of your data.

CREATE VIEW active_users AS SELECT * FROM users WHERE deleted_at IS NULL;
CREATE VIEW active_posts AS SELECT * FROM posts WHERE deleted_at IS NULL;

-- Example API query using the view
SELECT * FROM active_users WHERE id = 1;

-- Example of restoring a soft-deleted record
UPDATE users SET deleted_at = NULL WHERE id = 1;

Conclusion

Soft deletion is a powerful technique that can significantly enhance your data management strategy. It provides flexibility in data recovery, improves auditing capabilities, and can boost performance in large, interconnected datasets. While it does add some complexity to your database design and queries, the benefits often outweigh the costs for many applications.

Hope you learned something new today. Thanks, and please do share! ;)

A practical guide to using the JSONB type in PostgreSQL

JSONB, or JSON Binary, is a more efficient way to store and manipulate JSON data in PostgreSQL. It stores data in a decomposed binary format, which allows for faster access to individual elements within the JSON document. It is the recommended way to store JSON data in PostgreSQL. Why Use

WirekatMarkus

11 Things You Can Do to Secure Your Linux Server

Linux is one of the most popular and widely used operating systems in the world, especially for servers. Linux servers power millions of websites, applications, databases, and other services that we use every day. However, Linux servers are not immune to cyberattacks, and they require proper security measures to protect

WirekatMarkus

Optimizing SQL queries is essential for improving the performance of a database system. In PostgreSQL, like in any other relational database, the way data is structured and indexed plays a crucial role in query optimization. This article will outline the best practices for structuring data, indexing, and query optimization.

So let's start our journey with the basics! 😉

Optimizing SQL queries

Structuring Data

The foundation of query optimization begins with how the data is structured. Here are some best practices:

• Normalization: Ensure that your database design adheres to normalization rules up to 3NF to eliminate redundant data and ensure data integrity.
• Appropriate Data Types: Choose the most appropriate data types for each column. For example, use INTEGER or BIGINT for large numbers instead of VARCHAR.
• Use of Constraints: Apply constraints such as NOT NULL, UNIQUE, and CHECK to enforce data integrity and improve query performance.

Indexing

Indexes are critical for improving the performance of read operations. Here are some guidelines for effective indexing:

• Primary Key Index: Always define a primary key for each table; PostgreSQL automatically creates a unique index for it.
• Index Columns Used in WHERE Clauses: Create indexes on columns that are frequently used in WHERE clauses to speed up query execution.
• Partial Indexes: Use partial indexes when you only need to index a subset of rows, such as CREATE INDEX ON orders (order_date) WHERE status = 'SHIPPED';.
• Composite Indexes: For queries that filter on multiple columns, consider using composite indexes.

Avoiding Indexing Pitfalls

While indexes can improve performance, they can also lead to pitfalls if not used wisely:

• Over-Indexing: Creating too many indexes can slow down write operations, as each index needs to be updated on INSERT, UPDATE, or DELETE.
• Unused Indexes: Periodically review and remove indexes that are not used by the query planner.
• Index Maintenance: Rebuild indexes periodically to deal with bloat using REINDEX.

Indexing with Joined Tables

When dealing with joined tables, indexing becomes even more crucial:

• Foreign Key Indexes: Index foreign key columns to speed up joins and enforce referential integrity.
• Indexing Join Columns: Index columns that are used in JOIN conditions to improve join performance.

Example

Here’s an example of creating an index on a joined table:

-- Assuming we have two tables, orders and customers, joined on customer_id
CREATE INDEX idx_customer_id ON orders (customer_id);

-- A query that benefits from the above index
SELECT *
FROM orders
JOIN customers ON orders.customer_id = customers.id
WHERE customers.last_name = 'Smith';

In this example, the index idx_customer_id helps to quickly locate orders for customers with the last name ‘Smith’, making the join operation more efficient.

This section has laid the groundwork for structuring data and indexing in PostgreSQL. In the following sections, we will delve into using EXPLAIN ANALYZE to further optimize queries and table structures.

Using EXPLAIN ANALYZE for Query Optimization

The EXPLAIN ANALYZE command in PostgreSQL is a powerful tool for understanding how the query planner processes a query and identifying areas for optimization. Let’s explore how to use it effectively.

Basics of EXPLAIN ANALYZE

When you prepend EXPLAIN ANALYZE to a query, PostgreSQL provides a detailed execution plan along with actual runtime statistics. Here’s an example:

EXPLAIN ANALYZE
SELECT *
FROM orders
WHERE order_date BETWEEN '2024-01-01' AND '2024-03-31';

The output will show the query plan, estimated costs, and actual execution time for each step. Analyzing this output helps you identify bottlenecks and areas for improvement.

Example output

EXPLAIN ANALYZE
SELECT o.order_id, c.first_name, c.last_name
FROM orders o
JOIN customers c ON o.customer_id = c.id
WHERE o.order_date BETWEEN '2024-01-01' AND '2024-03-31';

Output:

QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------
Hash Join  (cost=100.00..200.00 rows=1000 width=64) (actual time=5.123..10.456 rows=500 loops=1)
  Hash Cond: (o.customer_id = c.id)
  ->  Seq Scan on orders o  (cost=0.00..50.00 rows=1000 width=32) (actual time=0.012..2.345 rows=1000 loops=1)
        Filter: ((order_date >= '2024-01-01'::date) AND (order_date <= '2024-03-31'::date))
        Rows Removed by Filter: 5000
  ->  Hash  (cost=50.00..50.00 rows=1000 width=32) (actual time=5.100..5.100 rows=1000 loops=1)
        Buckets: 1024  Batches: 1  Memory Usage: 64kB
        ->  Seq Scan on customers c  (cost=0.00..50.00 rows=1000 width=32) (actual time=0.010..2.345 rows=1000 loops=1)
Planning Time: 0.123 ms
Execution Time: 10.789 ms

Explanation:

• The query involves a hash join between the orders and customers tables.
• The estimated cost is around 100 units.
• The actual execution time is approximately 10.8 milliseconds.
• The filter condition on order_date is applied during the scan.
• The query planner chose a hash join strategy.

Query Optimization Examples

Let’s consider two scenarios: a simple query and a more complex one involving multiple joins and filters.

1. Simple Query Optimization: Suppose we have an orders table with an index on the order_date column. We want to retrieve orders placed between January 1, 2024, and March 31, 2024. The initial query might look like this:

SELECT *
FROM orders
WHERE order_date BETWEEN '2024-01-01' AND '2024-03-31';

Using EXPLAIN ANALYZE, we can verify that the index is being used efficiently. If not, we might need to reindex or consider other optimizations.

2. Complex Query Optimization: Imagine a more intricate scenario involving four joined tables: orders, customers, products, and order_items. We want to retrieve details of orders placed by customers with the last name ‘Smith’ for a specific product category. The query might look like this:

SELECT o.order_id, c.first_name, c.last_name, p.product_name
FROM orders o
JOIN customers c ON o.customer_id = c.id
JOIN order_items oi ON o.order_id = oi.order_id
JOIN products p ON oi.product_id = p.id
WHERE c.last_name = 'Smith'
  AND p.category = 'Electronics';

Using EXPLAIN ANALYZE, we can analyze the execution plan, check for sequential scans, and ensure that indexes are utilized efficiently. Adjusting indexes, rewriting the query, or denormalizing data may be necessary for optimal performance.

Practical Optimization

Consider the following steps when optimizing queries:

• Review Execution Plans: Regularly analyze execution plans to identify inefficiencies.
• Index Maintenance: Rebuild indexes periodically to prevent bloat.
• Analyze Real Data: Use real data and representative workloads for accurate optimization.

Why uppercase SQL is so common, and why it doesn’t make sense

SQL, or Structured Query Language, is a widely used language for interacting with databases. SQL allows you to create, manipulate, and query data in a structured and efficient way. HOWEVER, THERE IS ONE ASPECT OF SQL THAT OFTEN SPARKS DEBATE AMONG DEVELOPERS - WHICH CASE DO YOU USE ..AND WHY

WirekatMarkus

Understanding and Utilizing EXPLAIN ANALYZE Output

The EXPLAIN ANALYZE command in PostgreSQL provides valuable insights into how the query planner processes a query. By analyzing its output, you can identify bottlenecks, optimize query performance, and make informed decisions. Let’s dive into the details of reading and interpreting the EXPLAIN ANALYZE output.

Anatomy of EXPLAIN ANALYZE Output

When you execute a query with EXPLAIN ANALYZE, PostgreSQL generates an execution plan that consists of several components:

1. Plan Tree: The execution plan is represented as a tree structure. Each node in the tree corresponds to an operation (e.g., scan, join, filter) performed during query execution.
2. Cost Estimates: PostgreSQL estimates the cost of each operation based on factors like I/O, CPU, and memory usage.
3. Actual Execution Times: The actual time spent executing each operation during query execution.
4. Statistics: Information about the number of rows processed, memory usage, and other relevant metrics.

Common Output Columns

Let’s focus on some key columns you’ll encounter in the EXPLAIN ANALYZE output:

• Node Type: Describes the type of operation (e.g., Seq Scan, Index Scan, Hash Join).
• Relation Name: The name of the table or index being accessed.
• Startup Cost: The estimated cost before starting the operation.
• Total Cost: The total estimated cost for the entire query.
• Actual Rows: The actual number of rows processed.
• Actual Time: The actual time spent executing the operation.

Interpreting the Output

1. Sequential Scan (Seq Scan):
   • Indicates a full table scan.
   • Check if an index can be used instead.
   • Consider adding an index on frequently filtered columns.
   • Look for high Seq Scan costs.
2. Index Scan (Index Only Scan):
   • Indicates an index lookup.
   • Verify that the index is being used efficiently.
   • Check for index bloat and fragmentation.
3. Join Strategies (Nested Loop, Hash Join, Merge Join):
   • Understand the join type used.
   • Nested Loop: Good for small tables.
   • Hash Join: Suitable for large tables.
   • Merge Join: Requires sorted input.
4. Filter Conditions (Filter):
   • Identify expensive filter conditions.
   • Optimize WHERE clauses and JOIN conditions.
   • Use appropriate indexes.
5. Sort Operations (Sort):
   • Look for sorting steps in the plan.
   • Consider adding indexes or adjusting query order.

Using the Output for Optimization

1. Identify Costly Nodes:
   • Focus on nodes with high startup or total costs.
   • Optimize these operations first.
2. Check Index Usage:
   • Ensure indexes are being utilized.
   • Verify that the most selective index is chosen.
3. Analyze Actual Times:
   • Compare actual execution times to estimates.
   • Investigate significant discrepancies.
4. Experiment with Changes:
   • Rewrite queries, adjust indexes, or denormalize data.
   • Rerun EXPLAIN ANALYZE to validate improvements.

Example 1: Worst Possible EXPLAIN ANALYZE Output

Consider the following query:

EXPLAIN ANALYZE
SELECT *
FROM orders
WHERE order_date BETWEEN '2024-01-01' AND '2024-03-31';

Output (Worst Case):

Seq Scan on orders  (cost=0.00..15629.12 rows=230412 width=422) (actual time=13.024..13.024 rows=1000 loops=1)
  Filter: ((order_date >= '2024-01-01'::date) AND (order_date <= '2024-03-31'::date))
  Rows Removed by Filter: 5000
Planning Time: 0.123 ms
Execution Time: 13.024 ms

Explanation:

• The query performs a sequential scan on the entire orders table.
• The estimated cost is high (15629.12).
• The actual execution time is 13.024 milliseconds.
• The filter condition on order_date is applied during the scan.
• This is inefficient due to the lack of an appropriate index.
• Conclusion

Example 2: Best Optimized EXPLAIN ANALYZE Output

After creating an index on the order_date column:

CREATE INDEX idx_order_date ON orders (order_date);
ANALYZE;

Now let’s re-run the query:

EXPLAIN ANALYZE
SELECT *
FROM orders
WHERE order_date BETWEEN '2024-01-01' AND '2024-03-31';

Output (Best Case):

Index Scan using idx_order_date on orders  (cost=0.14..8.16 rows=1000 width=422) (actual time=0.012..0.012 rows=1000 loops=1)
  Index Cond: ((order_date >= '2024-01-01'::date) AND (order_date <= '2024-03-31'::date))
Planning Time: 0.123 ms
Execution Time: 0.012 ms

Explanation:

• The query now performs an index scan using the idx_order_date.
• The estimated cost is low (8.16).
• The actual execution time is significantly faster (0.012 ms).
• The index condition directly filters the relevant rows.

By optimizing the query with an appropriate index, we achieved a substantial improvement in execution time.

Mastering the art of reading EXPLAIN ANALYZE output empowers you to fine-tune your queries, optimize execution plans, and achieve better database performance. Regularly analyze query plans, experiment with changes, and keep learning from the output to become an effective query optimizer.

Refactoring a Poorly Optimized Table Structure

In this section, we’ll examine a poorly optimized table structure and the corresponding queries. We’ll then apply the knowledge from the previous sections to refactor the table structure, use indexes effectively, and optimize the queries with EXPLAIN ANALYZE.

Identifying the Issues

CREATE TABLE order_details (
    order_id INT,
    product_id INT,
    product_name VARCHAR(255),
    quantity INT,
    unit_price MONEY,
    customer_id INT,
    customer_name VARCHAR(255),
    order_date DATE
);

The table contains redundant data (product_name, customer_name), and there’s no clear primary key. Queries on this table are slow, especially those involving joins and filters.

Example of a Bad Query

Here’s an example of a query that performs poorly due to the table’s structure:

SELECT customer_name, SUM(quantity * unit_price) AS total_spent
FROM order_details
WHERE order_date BETWEEN '2024-01-01' AND '2024-03-31'
GROUP BY customer_name;

Without proper indexing, this query results in a full table scan, which is inefficient.

Applying Best Practices and Indexing

To improve the structure, we normalize the table and create appropriate indexes:

-- Normalized tables
CREATE TABLE customers (
    id SERIAL PRIMARY KEY,
    name VARCHAR(255)
);

CREATE TABLE products (
    id SERIAL PRIMARY KEY,
    name VARCHAR(255),
    price MONEY
);

CREATE TABLE orders (
    id SERIAL PRIMARY KEY,
    customer_id INT REFERENCES customers(id),
    order_date DATE
);

CREATE TABLE order_items (
    order_id INT REFERENCES orders(id),
    product_id INT REFERENCES products(id),
    quantity INT
);

-- Indexes
CREATE INDEX idx_order_date ON orders (order_date);
CREATE INDEX idx_customer_id ON orders (customer_id);

Optimizing the Query

Now, let’s rewrite the original query to utilize the new structure and indexes:

SELECT c.name, SUM(oi.quantity * p.price) AS total_spent
FROM customers c
JOIN orders o ON c.id = o.customer_id
JOIN order_items oi ON o.id = oi.order_id
JOIN products p ON oi.product_id = p.id
WHERE o.order_date BETWEEN '2024-01-01' AND '2024-03-31'
GROUP BY c.name;

By refactoring the table structure and applying indexing best practices, we’ve transformed a poorly optimized setup into one that supports efficient queries. This demonstrates the importance of thoughtful database design and the power of tools like EXPLAIN ANALYZE in optimizing SQL queries.

In the next section, we’ll discuss how to approach designing a database schema with optimization in mind from the start.

Designing a Database Schema with Performance Optimization in Mind

When starting the design of a database schema, it’s crucial to consider performance optimization from the outset. This proactive approach can save significant time and resources later on. Here are key considerations for designing an optimized database schema:

Understand the Data and its Usage

• Data Modeling: Begin with a clear model of the data entities, their relationships, and the business rules that govern them.
• Workload Analysis: Understand the types of queries that will be run against the database, their frequency, and their performance requirements.

Apply Normalization Prudently

• Normalization: Apply normalization principles to eliminate redundancy and ensure data integrity, but also consider where denormalization might be beneficial for performance.
• Balance: Find the right balance between normalization for data integrity and denormalization for query efficiency.

Indexing Strategy

• Selective Indexing: Plan your indexes based on the queries you expect to run. Not every column needs an index.
• Index Types: Understand the different types of indexes (B-tree, hash, GIN, GiST) and choose the one that fits the use case.

Design for Scalability

• Scalability: Design tables and relationships that can scale with the growth of data.
• Partitioning: Consider partitioning large tables to improve manageability and performance.

Anticipate Future Changes

• Flexibility: Allow for flexibility in your schema design to accommodate future changes without significant rework.
• Metadata: Store metadata effectively to aid in future optimizations and maintenance.

Use Constraints and Relationships Wisely

• Constraints: Implement constraints such as PRIMARY KEY, FOREIGN KEY, UNIQUE, and CHECK to enforce data integrity.
• Relationships: Define clear relationships between tables using foreign keys to ensure referential integrity and optimize joins.

Example

-- Products table with a primary key and price index
CREATE TABLE products (
    id SERIAL PRIMARY KEY,
    name VARCHAR(255) NOT NULL,
    category VARCHAR(255) NOT NULL,
    price NUMERIC(10, 2)
);
CREATE INDEX idx_products_price ON products (price);

-- Customers table with a primary key
CREATE TABLE customers (
    id SERIAL PRIMARY KEY,
    name VARCHAR(255) NOT NULL,
    email VARCHAR(255) UNIQUE NOT NULL
);

-- Orders table with foreign keys and an index on the order_date
CREATE TABLE orders (
    id SERIAL PRIMARY KEY,
    customer_id INT REFERENCES customers(id),
    order_date DATE NOT NULL,
    status VARCHAR(50)
);
CREATE INDEX idx_orders_order_date ON orders (order_date);

-- Order items table with a composite primary key
CREATE TABLE order_items (
    order_id INT REFERENCES orders(id),
    product_id INT REFERENCES products(id),
    quantity INT NOT NULL,
    PRIMARY KEY (order_id, product_id)
);

In this schema, we’ve considered the need for efficient querying by indexing critical columns and ensuring that relationships are properly defined with foreign keys. The structure is normalized, but with an eye towards the types of queries that will be run, ensuring that performance is a key consideration.

Designing a database schema with optimization in mind involves understanding the data, applying normalization judiciously, planning an indexing strategy, designing for scalability, anticipating future changes, and using constraints and relationships wisely. By considering these factors from the beginning, you can create a robust, efficient, and scalable database schema.

A practical guide to using the JSONB type in PostgreSQL

JSONB, or JSON Binary, is a more efficient way to store and manipulate JSON data in PostgreSQL. It stores data in a decomposed binary format, which allows for faster access to individual elements within the JSON document. It is the recommended way to store JSON data in PostgreSQL. Why Use

WirekatMarkus

Conclusion

As we wrap up our exploration of SQL optimization, it’s clear that the path to swift and efficient database queries is both an art and a science. We’ve delved into the intricacies of data structuring, the strategic use of indexes, and the insightful depths of EXPLAIN ANALYZE. These are more than just techniques; they’re the tools that empower us to craft queries that not only solve problems but do so with optimal performance.

Optimization is not a one-off task but a continuous process of refinement and learning. As databases grow and requirements evolve, so too must our approach to managing and querying them. Staying informed and adaptable is key to keeping our databases running smoothly.

Whether you’re a seasoned SQL pro or just starting out, remember that every query is an opportunity to apply these principles. By doing so, you’ll ensure that your databases are not just operational, but are performing at their best.

Let’s carry forward the lessons learned here, applying them to our future work with the confidence that comes from a solid foundation in SQL optimization.

JSONB, or JSON Binary, is a more efficient way to store and manipulate JSON data in PostgreSQL. It stores data in a decomposed binary format, which allows for faster access to individual elements within the JSON document. It is the recommended way to store JSON data in PostgreSQL.

Why Use JSONB in PostgreSQL?

• Indexing: JSONB supports GIN (Generalized Inverted Index) indexing, which can significantly speed up query performance.
• Query Performance: JSONB provides better performance for complex queries involving JSON data.
• Data Manipulation: JSONB allows for more advanced data manipulation capabilities, such as deleting keys from the JSON document.

Why uppercase SQL is so common, and why it doesn’t make sense

SQL, or Structured Query Language, is a widely used language for interacting with databases. SQL allows you to create, manipulate, and query data in a structured and efficient way. HOWEVER, THERE IS ONE ASPECT OF SQL THAT OFTEN SPARKS DEBATE AMONG DEVELOPERS - WHICH CASE DO YOU USE ..AND WHY

WirekatMarkus

JSONB Operations in PostgreSQL

Working with JSONB in PostgreSQL involves a set of operations that allow you to create, query, update, and manipulate JSONB data efficiently.

Creating JSONB Data

To store data in a JSONB format, you first need to create a table with a JSONB column. Here’s how you can create a table for storing user profiles:

CREATE TABLE user_profiles (
    id SERIAL PRIMARY KEY,
    profile JSONB NOT NULL
);

Inserting JSONB Data

Once you have a table with a JSONB column, you can insert data into it. Here’s an example of inserting a user profile with various attributes:

INSERT INTO user_profiles (profile)
VALUES ('{"name": "John Doe", "age": 30, "interests": ["coding", "chess"]}');

Selecting JSONB Data

Selecting data from a JSONB column is similar to selecting from any other data type. However, you have the added benefit of using JSONB operators and functions to query specific elements within the JSONB document. Here’s how you can select profiles of users who are interested in chess:

SELECT *
FROM user_profiles
WHERE profile @> '{"interests": ["chess"]}';

Updating JSONB Data

Updating JSONB data can be done using the jsonb_set function to change existing values, or the || operator to add new key-value pairs. Here’s how you can update a user’s age and add a new interest:

UPDATE user_profiles
SET profile = jsonb_set(profile, '{age}', '31') || '{"interests": ["puzzles"]}'
WHERE profile->>'name' = 'John Doe';

Deleting from JSONB Data

To remove a key and its value from a JSONB document, you can use the - operator. Here’s how to remove the ‘interests’ key from a user’s profile:

UPDATE user_profiles
SET profile = profile - 'interests'
WHERE profile->>'name' = 'John Doe';

Indexing JSONB Data

For performance optimization, especially with large datasets, you can create indexes on JSONB columns. Here’s how to create a GIN index on the profile column:

CREATE INDEX idxgin ON user_profiles USING GIN (profile);

Real-Life Example: Inventory Management System

An inventory management system needs to track a wide range of product details, which may change frequently. Using JSONB, you can store this dynamic information efficiently.

Schema Design with JSONB:

CREATE TABLE inventory (
    id SERIAL PRIMARY KEY,
    product_name VARCHAR(255) NOT NULL,
    product_details JSONB NOT NULL
);

Inserting Data with JSONB:

INSERT INTO inventory (product_name, product_details)
VALUES
('Widget', '{"manufacturer": "WidgetCo", "specs": {"weight": "1kg", "color": "blue"}}'),
('Gadget', '{"manufacturer": "GadgetCorp", "specs": {"dimensions": "5x2x8", "power": "battery-operated"}}');

Querying Data with JSONB: To find all widgets in blue color, you can use the following query:

SELECT product_name, product_details
FROM inventory
WHERE product_details @> '{"specs": {"color": "blue"}}';

Strategic Use of JSONB in PostgreSQL

JSONB is a versatile data type in PostgreSQL that is particularly useful when dealing with unstructured or semi-structured data. It shines in scenarios where the schema is not fixed, where data integrity is paramount, and where quick read access is required. Below are some specific use cases where JSONB is the preferred choice, along with additional examples to illustrate its practical applications.

Changelogs

In systems where tracking changes over time is crucial, such as version control or audit logs, JSONB is invaluable. By storing a JSONB dump of the changed row along with a diff from the old state, you can maintain a comprehensive changelog without the overhead of relational schema updates.

Example:

CREATE TABLE event_logs (
    thing_id INT,
    changed_at TIMESTAMP,
    state JSONB,
    diff JSONB
);

External API Data

When integrating with external APIs, the data structure may not be under your control and can change without notice. Storing API responses in a JSONB column ensures that all data is captured, regardless of its structure. Using a typed view allows for easy access to structured data while preserving the ability to adapt to changes.

Example:

CREATE VIEW typed_api_data AS
SELECT (data->>'amount')::integer
FROM api_data;

Search Parameters

For applications utilizing search engines like Elasticsearch, storing search parameters in JSONB allows for flexibility in the search criteria and the ability to quickly adapt to new search features or changes in the search engine’s API.

Example:

CREATE TABLE search_queries (
    query_id SERIAL PRIMARY KEY,
    parameters JSONB
);

Dynamic Configuration

JSONB is ideal for storing configuration parameters that may vary between environments or need to be updated on-the-fly without database migrations.

Example:

CREATE TABLE app_config (
    config_name VARCHAR(255) NOT NULL,
    config_values JSONB NOT NULL
);

User Preferences

Storing user preferences in JSONB allows for a personalized experience where each user can have a unique set of preferences without the need for a complex relational setup.

Example:

CREATE TABLE user_preferences (
    user_id INT REFERENCES users(id),
    preferences JSONB
);

To JSONB, or not?

When to Consider Using JSONB

• Dynamic Schema: JSONB can be useful when dealing with a schema that changes extremely frequently.
• Complex Data Structures: For deeply nested and complex data structures, JSONB is an option, but evaluate if this complexity can be reduced or better managed with a relational design.
• Full-Text Search: JSONB supports full-text search within JSON documents, but consider whether a dedicated search engine might better serve your needs.
• Data Integrity: JSONB ensures data integrity for JSON formats, but ask yourself if this data needs to be in JSON format or if it could be normalized.

When to Prefer Traditional Relational Models

• Simple, Flat Data: Traditional relational columns are more efficient for simple and flat data structures. They should be your default choice.
• Heavy Write Operations: The overhead of JSONB can slow down write-heavy applications. In such cases, a relational model is likely to perform better.
• Data Normalization: Relational models excel at avoiding data redundancy and ensuring data integrity through normalization. Use them whenever possible.

Important Considerations When Using JSONB in PostgreSQL

The use of JSONB in PostgreSQL should be approached with caution and a clear understanding of its implications. While JSONB offers flexibility for storing JSON data, it is not always the optimal choice for every situation.

Data Manipulation: Be aware that any modification to a JSONB value, no matter how small, necessitates rewriting the entire value to disk. This can lead to inefficiencies, particularly with large JSON documents.

Complexity: Frequent manipulation of complex structures or arrays within JSONB can become cumbersome and may result in less efficient operations compared to a normalized relational model.

Data Integrity: JSONB relinquishes the database’s ability to enforce data type checks. This means there is a risk of storing incorrect data types, such as entering a string where a date is expected.

Performance: While JSONB supports indexing which can aid in query performance, it is crucial to design your queries carefully. Inefficient queries can negate the benefits of indexing and lead to performance issues.

Use Case Appropriateness: JSONB is best used in scenarios where the schema is dynamic and subject to frequent changes. However, if your data is simple, flat, and does not require the flexibility offered by JSONB, traditional relational columns may be more suitable.

Normalization vs. Flexibility: A properly normalized model is often preferable for maintaining data integrity and avoiding redundancy. JSONB should be considered when normalization is not feasible or would lead to performance degradation.

Conclusion

JSONB in PostgreSQL is a powerful tool for developers dealing with dynamic and complex data structures.

It combines the flexibility of JSON with the performance benefits of a binary format, making it an excellent choice for modern applications that require efficient data storage and retrieval.

However, it should be employed when its advantages clearly outweigh the limitations. Developers must remain vigilant about the trade-offs, ensuring that the use of JSONB aligns with the application’s performance requirements and data integrity needs. By adopting a pragmatic approach to JSONB, one can harness its strengths while maintaining the robustness and efficiency of the database system.

Why uppercase SQL is so common, and why it doesn’t make sense

SQL, or Structured Query Language, is a widely used language for interacting with databases. SQL allows you to create, manipulate, and query data in a structured and efficient way. HOWEVER, THERE IS ONE ASPECT OF SQL THAT OFTEN SPARKS DEBATE AMONG DEVELOPERS - WHICH CASE DO YOU USE ..AND WHY

WirekatMarkus

Python, a language that has permeated the tech industry, is often praised for its simplicity and ease of use. Fields such as AI and Data Science seem to be all about it.

However, when we strip away the veneer of convenience, we are left with a language that presents numerous challenges, particularly in long-term development and large-scale projects.

The reason for this article? I've mostly used strict languages and moved to a project spearheaded by a Python development team. It can be quite a shock at how different the development process can be between say, Python and stricter JVM-focused languages, Go or C#. Please expect bias in this article!

Dynamic Typing = A Source of Silent Errors

Python’s dynamic typing is both a blessing and a curse. On one hand, it allows for flexibility and rapid development. On the other, it can lead to subtle, hard-to-detect errors. Imagine a large-scale project where a single untyped variable propagates through the codebase, causing a chain reaction of bugs.

These errors remain dormant until they suddenly explode at runtime, leaving developers scratching their heads. The lack of enforced type safety means that Python won’t catch these issues until it’s too late. While dynamic typing can enhance productivity, it also demands vigilance and thorough testing.

While dynamic typing grants flexibility, it also conceals lurking dangers. Consider this example:

def calculate_total(items):
    total = 0
    for item in items:
        total += item
    return total

# Usage
shopping_cart = [10, 20, "30", 40]
print(calculate_total(shopping_cart))  # Oops! Concatenation instead of addition

In this case, the dynamic typing allowed the string "30" to silently slip into our numeric calculation, resulting in unexpected behavior.

Simplicity?

Python’s simple syntax is often praised as its hallmark. It encourages readability and conciseness, making it an excellent choice for beginners. However, this simplicity can be deceptive.

Novice developers, drawn in by Python’s elegance, may fall into the trap of writing code that “works” but lacks structural integrity. They may prioritize quick solutions over long-term maintainability.

The code passes initial tests, but when deployed in a production environment, it reveals its flaws. The illusion of simplicity can lead to costly mistakes, especially as projects evolve and requirements change.

Novice developers often fall into the trap of writing code that “works” but lacks structural integrity. Take this snippet:

def divide(a, b):
    return a / b

result = divide(10, 0)  # Division by zero – a runtime disaster

The code passes initial tests, but in production, it crashes. The illusion of simplicity can lead to costly mistakes.

Scalability Concerns

As applications grow, Python’s robustness issues become apparent. Memory management, concurrency, and performance all come under scrutiny. Python’s memory footprint can balloon unexpectedly, impacting server resources. Its Global Interpreter Lock (GIL) restricts true parallelism, hindering multi-threaded applications.

For high-traffic systems, Python’s limitations become glaring. While it excels in scripting and prototyping, it struggles to keep pace with the demands of large-scale, production-grade systems.

Languages like Go and Java, designed with scalability in mind, offer better solutions for such scenarios.

As projects grow, Python’s robustness issues emerge. Memory management and concurrency become bottlenecks. For instance:

# Inefficient memory usage
def factorial(n):
    if n == 0:
        return 1
    else:
        return n * factorial(n - 1)

# Try factorial(1000) – stack overflow!

Python’s recursive approach consumes excessive memory, making it unsuitable for high-traffic systems.

Legacy of GIL

The Global Interpreter Lock (GIL) is a relic from Python’s past that continues to haunt its present. Originally introduced for thread safety, the GIL ensures that only one thread executes Python bytecode at a time.

While this simplifies memory management, it severely limits multi-core utilization. In a world where multi-core processors are ubiquitous, Python’s GIL becomes a bottleneck.

Developers resort to workarounds like multi-processing, which introduces complexity and overhead. The GIL persists as a legacy issue, frustrating those who seek true parallelism.

import threading

def perform_task():
    # Intensive task here

threads = [threading.Thread(target=perform_task) for _ in range(10)]
for thread in threads:
    thread.start()
for thread in threads:
    thread.join()
# Despite creating threads, the GIL limits true parallelism

Inconsistent Language Design

Python’s design philosophy is a double-edged sword. While it encourages creativity and multiple ways to achieve a goal, it also leads to inconsistency.

Different libraries may adopt divergent conventions, leaving developers puzzled. Consider string formatting: f-strings, .format(), and concatenation are all valid approaches. This lack of uniformity affects code readability and maintainability.

In contrast, languages like Kotlin and Java adhere to stricter guidelines, resulting in cleaner, more predictable codebases.

As an example, multiple ways to achieve the same result lead to confusion:

# String formatting options
name = "Alice"
greeting = f"Hello, {name}!"
# Or: greeting = "Hello, {}!".format(name)
# Or: greeting = "Hello, " + name + "!"

This lack of uniformity hampers code readability and maintainability. It isn't even readable syntax compared to, say, Kotlin.

Dependency Hell

Python’s package management can be a labyrinth. The ease of installing third-party packages comes with a price: dependency conflicts. As projects accumulate dependencies, maintaining compatibility becomes challenging. Different packages may require conflicting versions of the same library.

Resolving these conflicts involves detective work, trial and error, and sometimes compromises. The infamous “dependency hell” can trap even seasoned developers, leading to frustration and wasted hours. It's almost as bad as JavaScript.

Conflicting dependencies create a tangled web:

# Two packages requiring different versions of the same dependency
import package_a  # Requires library X v1.0
import package_b  # Requires library X v2.0
# Result: Unpredictable behavior or runtime errors

Lack of Modern Features

Python, despite its popularity, often lags behind in adopting modern programming features. While recent versions introduced type hints and pattern matching, other languages had these features long ago.

For example, Scala’s pattern matching has been a staple for years. Python’s slow adoption rate can hinder innovation and productivity, especially when developers crave expressive tools to tackle complex problems.

# Python 3.10 introduced pattern matching
match value:
    case 0:
        print("Zero")
    case _:
        print("Non-zero")
# But languages like Scala had this years ago

Poor Refactoring Support

Refactoring Python code is akin to untangling a web of spaghetti. Its dynamic nature makes automated refactoring tools cautious. Without strict types, these tools struggle to predict the impact of changes accurately.

# Refactoring this function is risky
def process_data(data):
    # Process data here

# Manual review and testing become essential

As a result, developers often rely on manual review and extensive testing during refactoring. The lack of robust refactoring support hampers maintainability and increases the risk of introducing new bugs.

Summary

It becomes clear that Python, while accessible and popular for certain applications, is fraught with challenges that can hinder development, especially as projects grow in complexity and size.

In summary, while Python may offer some convenience features like Django Admin and ease of use in AI, these do not compensate for its fundamental flaws. For robust backend development, developers are better served by languages that offer strict typing, better performance, and a more coherent design philosophy. Python, with its messy design and lack of efficiency, is often not the wisest choice for serious development endeavors.

For those seeking a more reliable and scalable solution, languages like Kotlin, Java, or Go offer a more structured and performance-oriented approach, making them preferable choices for serious backend development.

Ever been greeted by a "404 Not Found" message while browsing the web? It's one of the many status codes used by the web. 404 is self-explanatory - usually an indication that a resource or page was not found.

The (almost) interesting history of Port Numbers

Port numbers are the numerical identifiers that allow computers to communicate over the internet. They are assigned to different applications and protocols, such as web browsing, email, file transfer, and so on. But how did these port numbers come to be? Who decided which port number belongs to which service?

WirekatMarkus

HTTP status codes are the numerical messages that web servers send back to browsers or other clients, indicating the outcome of a request. They are part of the HTTP protocol, which stands for Hypertext Transfer Protocol, the standard way of communicating on the web.

HTTP status codes are divided into five categories, based on the first digit of the code:

• 1xx: Informational responses, indicating that the request has been received and is being processed.
• 2xx: Success responses, indicating that the request has been successfully completed.
• 3xx: Redirection responses, indicating that the request needs to be redirected to another location or resource.
• 4xx: Client error responses, indicating that the request is invalid or cannot be fulfilled by the server.
• 5xx: Server error responses, indicating that the server encountered an error or is unable to handle the request.

But how did these codes come to be? Who invented them, and why? And what are some of the most mysterious or obscure codes that exist? In this article, we will explore the history of HTTP status codes, from their origins to their present-day usage.

The Origins of HTTP and Status Codes

The HTTP protocol was invented by Tim Berners-Lee, a British computer scientist who is widely regarded as the father of the World Wide Web. Berners-Lee was working at CERN, the European Organization for Nuclear Research, in the late 1980s and early 1990s, when he came up with the idea of creating a system that would allow researchers to share and access information across different computers and networks.

Berners-Lee designed the first version of HTTP, along with the concepts of URLs (Uniform Resource Locators), HTML (Hypertext Markup Language), and web servers and browsers. He also wrote the first web browser and web server software, and launched the first website in 1991.

Guide to using the NGINX web server and reverse proxy

Nginx is a popular web server and reverse proxy that can handle high-performance and high-concurrency web applications. In this tutorial blog post, I will show you how to install and configure nginx on Ubuntu and RHEL-derivatives, how to use different conventions for managing nginx configuration files, and how to use

WirekatMarkus

The first version of HTTP, known as HTTP/0.9, was very simple and had only one method: GET. This method allowed clients to request a resource from a server, such as a web page or an image. The server would respond with either the requested resource, or an error message. There were no status codes in HTTP/0.9, only plain text messages.

For example, if a client requested a web page that existed on the server, the server would respond with something like:

<HTML>
<HEAD>
<TITLE>Example Page</TITLE>
</HEAD>
<BODY>
<H1>Hello, World!</H1>
</BODY>
</HTML>

If the client requested a web page that did not exist on the server, the server would respond with something like:

Bad request

As you can see, HTTP/0.9 was very limited and did not provide much information about the status of the request or the response. It also did not support other methods, such as POST, PUT, DELETE, or HEAD, which are used to send data, update or delete resources, or get metadata from the server.

The introduction of status codes & headers

HTTP/0.9 soon proved to be inadequate for the growing needs and complexity of the web. Therefore, Berners-Lee and other developers started working on a new version of HTTP, which would introduce more features and functionality. This version, known as HTTP/1.0, was officially released in 1996 as RFC 1945, a document that defines the specifications and standards of the protocol.

One of the major improvements of HTTP/1.0 was the introduction of status codes. Status codes are three-digit numbers that indicate the outcome of a request, and are accompanied by a short textual phrase that describes the meaning of the code. Status codes are intended to provide more information and feedback to the clients and the developers, and to enable more efficient and reliable communication between the servers and the clients.

The first version of HTTP/1.0 defined 14 status codes, divided into three categories:

• 2xx: Successful responses, indicating that the request has been successfully completed. The only code in this category was 200 OK, which means that the server has fulfilled the request and returned the requested resource.
• 3xx: Redirection responses, indicating that the request needs to be redirected to another location or resource. There were six codes in this category, such as 301 Moved Permanently, which means that the requested resource has been permanently moved to a new URL, or 304 Not Modified, which means that the requested resource has not changed since the last request and can be cached by the client.
• 4xx: Client error responses, indicating that the request is invalid or cannot be fulfilled by the server. There were seven codes in this category, such as 400 Bad Request, which means that the request is malformed or contains invalid syntax, or 404 Not Found, which means that the requested resource does not exist on the server.

HTTP/1.0 also introduced the concept of headers, which are additional information that can be sent along with the request or the response. Headers can provide metadata, such as the content type, the content length, the date, the server name, the client name, the cookies, the cache control, and so on. Headers can also affect the behavior of the request or the response, such as redirecting the client, negotiating the content format, or requiring authentication.

For example, a typical HTTP/1.0 request and response might look something like this:

GET /example.html HTTP/1.0
User-Agent: Mozilla/5.0
Host: www.example.com
Accept: text/html

HTTP/1.0 200 OK
Date: Sat, 17 Feb 2024 10:19:52 GMT
Server: Apache/2.4.46
Content-Type: text/html
Content-Length: 1024

<HTML>
<HEAD>
<TITLE>Example Page</TITLE>
</HEAD>
<BODY>
<H1>Hello, World!</H1>
</BODY>
</HTML>

HTML Tutorial for Beginners

HTML stands for HyperText Markup Language. It is the standard language for creating web pages and web applications. HTML describes the structure and content of a web page using tags and attributes. Tags are keywords that define different elements in a web page, such as headings, paragraphs, images, links, etc.

WirekatMarkus

Further evolution - HTTP/1.1

HTTP/1.0 was a significant improvement over HTTP/0.9, but it still had some limitations and drawbacks. For instance, HTTP/1.0 did not support persistent connections, which means that each request and response required opening and closing a new TCP connection, which was inefficient and costly. HTTP/1.0 also did not support pipelining, which means that each request had to wait for the previous response to finish, which was slow and wasteful. HTTP/1.0 also did not support chunked encoding, which means that the server had to know the size of the response before sending it, which was impractical and restrictive.

Therefore, another version of HTTP was developed, which would address these issues and add more features and functionality. This version, known as HTTP/1.1, was officially released in 1997 as RFC 2068, and later updated in 1999 as RFC 2616. HTTP/1.1 is the most widely used version of HTTP today, and is considered the standard version of the protocol.

One of the major improvements of HTTP/1.1 was the support for persistent connections, also known as keep-alive connections. This means that the TCP connection between the client and the server can be reused for multiple requests and responses, without having to open and close it for each one. This reduces the overhead and latency of the communication, and improves the performance and efficiency of the web.

HTTP/1.1 also introduced the support for pipelining, which means that the client can send multiple requests without waiting for the previous response to finish. This allows the client to make better use of the bandwidth and the connection, and reduces the waiting time and the blocking of the communication.

Another new feature was chunked encoding, which means that the server can send the response in chunks, without having to know the size of the response beforehand. This allows the server to stream the response dynamically, and to send data as soon as it is available, without having to buffer it or delay it.

This version also added new request types - OPTIONS, TRACE, CONNECT, and PATCH, which are used to query the server, test the connection, establish a tunnel, or apply partial updates to resources. HTTP/1.1 also added more headers, such as Connection, Transfer-Encoding, Host, Expect, and Range, which are used to control the connection, the encoding, the host name, the expectations, and the partial requests.

HTTP/1.1 also added more status codes, bringing the total number to 41. The new status codes were divided into four categories:

• 1xx: Informational responses, indicating that the request has been received and is being processed. There were eight codes in this category, such as 100 Continue, which means that the client should continue with the request, or 101 Switching Protocols, which means that the server agrees to switch to a different protocol requested by the client.
• 2xx: Success responses, indicating that the request has been successfully completed. There were five codes in this category, such as 201 Created, which means that the server has created a new resource as a result of the request, or 206 Partial Content, which means that the server has returned only a part of the resource requested by the client.
• 4xx: Client error responses, indicating that the request is invalid or cannot be fulfilled by the server. There were 18 codes in this category, such as 401 Unauthorized, which means that the request requires authentication, or 418 I’m a teapot, which is an Easter egg code that means that the server is a teapot and cannot brew coffee.
• 5xx: Server error responses, indicating that the server encountered an error or is unable to handle the request. There were 10 codes in this category, such as 500 Internal Server Error, which means that the server has encountered an unexpected condition that prevents it from fulfilling the request, or 503 Service Unavailable, which means that the server is temporarily unable to handle the request due to overload or maintenance.

Moving to the modern age - HTTP/2

HTTP/1.1 was a major improvement over HTTP/1.0, and it solved many of the problems and limitations of the previous version. However, HTTP/1.1 still had some challenges and drawbacks. For instance, HTTP/1.1 did not support multiplexing, which means that multiple requests and responses could not share the same connection, which could cause head-of-line blocking, which means that one slow request or response could delay the others. HTTP/1.1 also did not support server push, which means that the server could not send resources to the client without the client requesting them, which could improve the performance and user experience of the web.

Therefore, another version of HTTP was developed, which would address these issues and add more features and functionality. This version, known as HTTP/2, was officially released in 2015 as RFC 7540, and is based on an experimental protocol called SPDY, which was created by Google. HTTP/2 is the latest version of HTTP, and is gradually being adopted by more and more web servers and browsers.

One of the major improvements of HTTP/2 is the support for multiplexing, which means that multiple requests and responses can be sent and received concurrently over the same connection, using streams, frames, and priorities. This reduces the overhead and latency of the communication, and eliminates the head-of-line blocking problem.

HTTP/2 also introduced the support for server push, which means that the server can send resources to the client before the client requests them, using push promises. This allows the server to anticipate the client’s needs, and to improve the performance and user experience of the web.

HTTP/2 also added more features, such as binary encoding, which means that the headers and the frames are encoded in binary format, instead of plain text, which reduces the size and the complexity of the messages. HTTP/2 also added more security, such as mandatory TLS, which means that the connection between the client and the server must be encrypted, using Transport Layer Security.

HTTP/2 also added more status codes, bringing the total number to 63. The new status codes were divided into five categories:

• 1xx: Informational responses, indicating that the request has been received and is being processed. There were no new codes in this category, but the existing codes were redefined to apply to HTTP/2 streams, instead of HTTP/1.1 connections.
• 2xx: Success responses, indicating that the request has been successfully completed. There were two new codes in this category, such as 226 IM Used, which means that the server has fulfilled the request and the response is a representation of the result of one or more instance manipulations applied to the current instance, or 208 Already Reported, which means that the server has already fulfilled the request and the response is a list of bindings that have already been reported in previous responses.
• 3xx: Redirection responses, indicating that the request needs to be redirected to another location or resource. There were two new codes in this category, such as 308 Permanent Redirect, which means that the requested resource has been permanently moved to a new URL, and the client should use that URL for future requests, or 307 Temporary Redirect, which means that the requested resource has been temporarily moved to a new URL, and the client should use that URL for this request only.
• 4xx: Client error responses, indicating that the request is invalid or cannot be fulfilled by the server. There were four new codes in this category, such as 429 Too Many Requests, which means that the client has sent too many requests in a given period of time, and the server is throttling the requests, or 451 Unavailable For Legal Reasons, which means that the server is denying access to the resource due to legal demands or censorship.
• 5xx: Server error responses, indicating that the server encountered an error or is unable to handle the request. There were four new codes in this category, such as 511 Network Authentication Required, which means that the client needs to authenticate with the network before accessing the resource, or 508 Loop Detected, which means that the server has detected an infinite loop while processing the request.

The future of the web - HTTP/3

HTTP/2 was a major improvement over HTTP/1.1, and it solved many of the challenges and drawbacks of the previous version. However, HTTP/2 still had some limitations and drawbacks. For instance, HTTP/2 did not support UDP, which means that the protocol still relied on TCP, which is a reliable but slow and heavy transport layer protocol, which could cause congestion, packet loss, and retransmissions. HTTP/2 also did not support QUIC, which is a new transport layer protocol that combines the features of TCP, UDP, and TLS, and provides faster, more secure, and more reliable communication.

Therefore, another version of HTTP is being developed, which will address these issues and add more features and functionality. This version, known as HTTP/3, is still in draft stage, and is based on an experimental protocol called QUIC, which was also created by Google. HTTP/3 is the future version of HTTP, and is expected to be released in the near future.

One of the major improvements of HTTP/3 is the support for UDP, which means that the protocol will use User Datagram Protocol instead of Transmission Control Protocol as the transport layer protocol. UDP is a fast and lightweight protocol that does not guarantee reliability or order of the packets, but allows for more flexibility and efficiency of the communication.

HTTP/3 also introduced the support for QUIC, which means that the protocol will use Quick UDP Internet Connections as the application layer protocol. QUIC is a new protocol that combines the features of TCP, UDP, and TLS, and provides faster, more secure, and more reliable communication. QUIC also supports multiplexing, server push, binary encoding, and mandatory encryption, like HTTP/2, but with improved performance and robustness.

HTTP/3 also added more features, such as 0-RTT, which means that the client and the server can exchange data without any round-trip time, using session resumption and early data. This reduces the latency and improves the user experience of the web.

HTTP/3 also added more status codes, bringing the total number to 65. The new status codes were divided into five categories:

• 1xx: Informational responses, indicating that the request has been received and is being processed. There was one new code in this category, which is 103 Early Hints, which means that the server is sending some preliminary information, such as headers or links, before the final response.
• 2xx: Success responses, indicating that the request has been successfully completed. There were no new codes in this category, but the existing codes were redefined to apply to HTTP/3 streams, instead of HTTP/2 frames.
• 3xx: Redirection responses, indicating that the request needs to be redirected to another location or resource. There were no new codes in this category, but the existing codes were redefined to apply to HTTP/3 streams, instead of HTTP/2 frames.
• 4xx: Client error responses, indicating that the request is invalid or cannot be fulfilled by the server. There were no new codes in this category, but the existing codes were redefined to apply to HTTP/3 streams, instead of HTTP/2 frames.
• 5xx: Server error responses, indicating that the server encountered an error or is unable to handle the request. There were no new codes in this category, but the existing codes were redefined to apply to HTTP/3 streams, instead of HTTP/2 frames.

HTTP/3 is the future version of HTTP, and is expected to be released in the near future.

Introduction to REST APIs - What is a REST API? How to use or build a REST API?

What is a REST API? A REST API is a way of designing and implementing web services that follow the principles of REpresentational State Transfer (REST). REST is an architectural style that was proposed by Roy Fielding in his doctoral dissertation in 2000. REST is not a standard or a

WirekatMarkus

The Origins Behind Some Status Codes

HTTP status codes are not only useful and informative, but also interesting and entertaining. Some status codes have fascinating arguments or stories behind them, and some status codes are humorous or mysterious.

200 OK

The most common and familiar status code is 200 OK, which means that the server has fulfilled the request and returned the requested resource. This code is also the oldest and the simplest status code, as it was the only code in the 2xx category in HTTP/0.9.

418 I’m a teapot

One of the most humorous and mysterious status codes is 418 I’m a teapot, which means that the server is a teapot and cannot brew coffee. This code is not part of the official HTTP standard, but it was defined as an April Fools’ joke in 1998 by the Hyper Text Coffee Pot Control Protocol (HTCPCP), which is an extension of HTTP for controlling, monitoring, and diagnosing.. coffee pots?

The origin of the code is a reference to a children’s book called The Little Teapot, which is about a teapot that is sad because it cannot brew coffee, but then learns that it can do other things, such as whistle and pour tea. The code is also a parody of the Internet of Things (IoT), which is the concept of connecting everyday objects and devices to the internet and enabling them to communicate and exchange data.

The code is not meant to be taken seriously, but it has been implemented by some web servers and frameworks as an Easter egg or a joke. 

451 Unavailable For Legal Reasons

Another interesting and controversial status code is 451 Unavailable For Legal Reasons, which means that the server is denying access to the resource due to legal demands or censorship.

The origin of the code is a reference to a novel called Fahrenheit 451, which is written by Ray Bradbury and published in 1953. The novel is about a dystopian society where books are banned and burned by the government, and the title refers to the temperature at which paper ignites. The novel is a critique of censorship and authoritarianism, and a defense of freedom of expression and literature.

The code is meant to be used when the server is legally obliged to block or remove the resource, such as due to court orders, intellectual property rights, privacy laws, or national security. The code is also meant to inform the user of the reason and the authority behind the restriction, and to provide a link to more information or a way to challenge the decision.

The code is not widely used, but it has been implemented by some web servers and frameworks as a way to protest or raise awareness of censorship and internet freedom.

420 Enhance Your Calm

One of the most humorous and mysterious status codes is 420 Enhance Your Calm, which means that the client has sent too many requests and needs to calm down. This code is not part of the official HTTP standard, but it was used by the initial version of the Twitter Search and Trends API, which is an API that allows developers to access and analyze Twitter data.

The origin of the code is a reference to the movie 10 Things I Hate About You, which is a romantic comedy film released in 1999. In the movie, there is a scene where a character named Kat is angry and frustrated, and another character named Patrick tries to calm her down by saying "Just calm down. Relax. Take it easy. Enhance your calm."

The code is also a reference to the number 420, which is a slang term for cannabis or marijuana, and is often associated with the subculture of cannabis users.

The code is not meant to be taken seriously, but it was used by Twitter to limit the number of requests that a client can make to the API, and to prevent abuse or overload of the service. The code was later replaced by the standard code 429 Too Many Requests, which has the same meaning but is more formal and less humorous.

402 Payment Required

Another intriguing and mysterious status code is 402 Payment Required, which means that the server requires payment to access the resource. This code is part of the official HTTP standard, but it was never fully defined or implemented by the RFC 1945.

The origin of the code is not very clear, but it is likely that it was intended for directing the user to complete digital payments, such as using credit cards, online wallets, or cryptocurrencies. However, the code was never used for this purpose, as there were no standard or secure ways to handle online payments at the time, and there were other methods and protocols that emerged later, such as SSL, TLS, HTTPS, and OAuth.

The code is not used, but it has been reserved for future use, in case there is a need or a demand for it. The code is also used by some web servers and frameworks as an Easter egg or a joke.

530 Site is frozen

Another humorous and mysterious status code is 530 Site is frozen, which means that the site has been frozen due to inactivity or non-payment. This code is not part of the official HTTP standard, but it was used by some web servers.

The code is not widely used, but it has been implemented by some web servers and frameworks as a way to block or suspend the sites that are inactive or unpaid, such as free or trial sites, expired or cancelled sites, or abandoned or neglected sites.

Conclusion

I hope you've learned something new about the HTTP protocol, as well as discovered some obscure HTTP codes - even if some of them aren't in the official standard. Easter eggs are always fun!

I hope you enjoyed!

Understanding HTTP Cookie flags

Cookies? Cookie flags? In this article, we try to understand these attributes and see how they work.

WirekatMarkus

Half-Life is a classic game that has spawned countless mods, some of which have become standalone games themselves. But among the many fan-made creations, there is one that stands out for its originality, quality, and atmosphere: Half-Life: The World Machine.

Delta Particles: The Ultimate Half-Life Mod You Need to Play

If you’re a fan of Half-Life, the classic FPS game that revolutionized the genre, you might think you’ve seen and played it all. But what if I told you there’s a mod that transforms Half-Life into a whole new game, with new levels, weapons, enemies, characters, and

WirekatMarkus

The World Machine's plot

Half-Life: The World Machine is a mod that was released on October 22, 2023 by Dr.T, a German modder who also created Uriel, another Half-Life mod. The World Machine is a short but intense adventure that puts you in the shoes of Flynn Weißhaupt, a computer engineer of Black Mesa who worked on the Gibson, a powerful supercomputer.

Now with the Resonance Cascade, it is your mission to retrieve the data of the Gibson and destroy it, because the Gibson is too powerful to fall into the hands of alien forces or the military.

Gameplay & features

The mod features about an hour of gameplay, with a mix of puzzles, combat, and exploration. The level design is impressive, as you traverse the hidden and mysterious areas of Black Mesa, such as the underground labs, the ventilation shafts, and the secret passages.

The mod also has a distinctive aesthetic, with a dark and gritty tone, enhanced by the custom textures, sounds, and music.

The mod also has some surprises and twists along the way, as you discover the true nature of the Gibson and its connection to the events of Half-Life.

Linux compatibility

It works flawlessly on Linux under Wine, as do most Half-Life mods. Give it a try!

Conclusion

If you are a fan of Half-Life and its mods, you should definitely check out Half-Life: The World Machine. It is a hidden gem of Black Mesa that will challenge you, entertain you, and immerse you in a new and exciting story. 

You can download the mod for free from ModDB, and enjoy the adventure of Flynn Weißhaupt and the Gibson.

Half-Life: The World Machine mod

You are Flynn Weißhaupt, a computer engineer of Black Mesa who worked on the Gibson, a powerful supercomputer. Now with the Resonance Cascade, it is your mission to retrieve the data of the Gibson and destroy it. Because The Gibson is too powerful to…

ModDBDr.T

Half-Life 2 RTX: A Fan-Made Project to Revive a Classic with Ray Tracing

Half-Life 2 is one of the most acclaimed games of all time, but it’s also nearly 20 years old. That’s why a fan studio called Orbifold Studios is working on a project to update the game’s graphics with ray tracing and improved textures, using Nvidia’s modding tools and platform. The

WirekatMarkus

Port numbers are the numerical identifiers that allow computers to communicate over the internet. They are assigned to different applications and protocols, such as web browsing, email, file transfer, and so on.

But how did these port numbers come to be?

Who decided which port number belongs to which service?

And are there any interesting stories behind them?

In this article, we will explore the origins and evolution of port numbers, and reveal some of the hidden meanings and mysteries behind them.

How to Port Scan on Linux using nmap - a Powerful Tool for Network Exploration and Security Auditing

Nmap, which stands for Network Mapper, is a free and open-source tool that can scan networks and discover hosts, services, vulnerabilities, and other information. Nmap is widely used by network administrators, security professionals, hackers, and enthusiasts for various purposes, such as network inventory, security assessment, penetration testing, and troubleshooting. In

WirekatMarkus

Everything you need to know about Selfhosting on Linux

Selfhosting is the practice of running your own web services on your own hardware, instead of relying on third-party providers. Selfhosting can give you more control, privacy, and customization over your online presence, as well as save you money in the long run. However, selfhosting also comes with some challenges,

WirekatMarkus

The Early Days of Port Numbers

The concept of port numbers dates back to the early days of the internet, when it was still called the ARPANET. The ARPANET was a network of computers that connected research institutions and universities in the United States, funded by the Department of Defense. It was the precursor of the modern internet, and the first network to use the TCP/IP protocol suite.

TCP/IP

The TCP/IP protocol suite is a set of rules and standards that govern how data is transmitted and received over the internet. It consists of four layers: the application layer, the transport layer, the internet layer, and the network access layer.

The port numbers are part of the transport layer, and they are used by two main protocols: TCP and UDP. TCP stands for Transmission Control Protocol, and it is a reliable and ordered way of sending and receiving data. TCP ensures that the packets are delivered without errors, and in the correct sequence. UDP stands for User Datagram Protocol, and it is a fast and simple way of sending and receiving data. UDP does not guarantee that the packets are delivered without errors, or in the correct order, but it is more efficient and suitable for real-time applications, such as video streaming or online gaming.

Everything you need to know about using Git

Git is a distributed version control system that allows developers to track changes in their code, collaborate with others, and manage different versions of their projects. Git is widely used in software development, and knowing how to use it effectively can make your work easier and more productive. In this

WirekatMarkus

11 Things You Can Do to Secure Your Linux Server

Linux is one of the most popular and widely used operating systems in the world, especially for servers. Linux servers power millions of websites, applications, databases, and other services that we use every day. However, Linux servers are not immune to cyberattacks, and they require proper security measures to protect

WirekatMarkus

Origin of port numbers

The port numbers range from 0 to 65535, and they are divided into three categories: well-known ports, registered ports, and dynamic ports.

The well-known ports are the ones from 0 to 1023, and they are reserved for the most common and standardized services and protocols, such as HTTP, FTP, SMTP, DNS, etc.

The registered ports are the ones from 1024 to 49151, and they are assigned to specific applications and protocols by the Internet Assigned Numbers Authority (IANA), which is the organization that oversees the allocation and management of port numbers, IP addresses, and other internet resources.

The dynamic ports are the ones from 49152 to 65535, and they are used for temporary and random purposes, such as client-server communication, peer-to-peer networking, or testing.

The first official list of port numbers was published in 1972, in RFC 322, which stands for Request for Comments. RFCs are the documents that describe the specifications and standards of the internet, and they are written and reviewed by experts and researchers from the internet community. The list contained 23 port numbers, assigned to the following services and protocols:

Some of the numbers are quite obscure

Some of these services and protocols are still widely used today, such as HTTP, FTP, SMTP, and DNS. Others are obsolete or rarely used, such as RJE, RLP, and FINGER. Some of them have interesting names and functions, such as:

• TCPMUX: This stands for TCP Port Service Multiplexer, and it is a service that allows multiple applications to share the same port number, by using a special protocol to multiplex and demultiplex the data. For example, if two applications want to use port 80, they can use TCPMUX to send and receive data on port 1, and then use a sub-port number to identify the actual application. TCPMUX is rarely used today, as it is considered insecure and inefficient.
• ECHO: This is a service that simply echoes back whatever data it receives. It is used for testing and debugging purposes, to check if the connection is working properly. For example, if you send the message “Hello, world!” to port 7, you will receive the same message back.
• QUOTE: This is a service that returns a random quote of the day, from a database of famous or humorous sayings. It is used for entertainment or inspiration purposes, or as a source of fortune cookies. For example, if you send a request to port 17, you might receive a quote like “Be the change that you wish to see in the world. - Mahatma Gandhi” or “Never trust a computer you can’t throw out a window. - Steve Wozniak”.
• CHARGEN: This stands for Character Generator, and it is a service that generates a stream of random characters, such as letters, numbers, symbols, etc. It is used for testing and measuring the performance of the network, by sending and receiving large amounts of data. For example, if you connect to port 19, you might receive a stream of characters like “4fU7%$#n@!2h*”.
• FINGER: This is a service that allows you to query information about a user on a remote computer, such as their name, email, phone number, login time, etc. It is used for finding and contacting people on the network, or for spying on them. For example, if you send the username “alice” to port 79, you might receive information like “Alice Smith, alice@foo.com, +1 555 1234, logged in since 10:15 AM”.

The Evolution of Port Numbers

As the internet grew and evolved, so did the port numbers. New services and protocols were developed and introduced, and new port numbers were assigned to them by the IANA.

Some of the port numbers were chosen based on logical or technical reasons, such as compatibility, efficiency, or security. Others were chosen based on arbitrary or whimsical reasons, such as personal preference, humor, or coincidence.

Here are some examples of port numbers with original origins:

• Port 666: This is the port number for DOOM, which is a video game that allows you to shoot and kill demons in a hellish environment. DOOM was created in 1993 by id Software, a company that pioneered the first-person shooter genre. DOOM was assigned port 666 because it was the number of the beast, and because it was a joke by the developers, who wanted to scare and annoy the network administrators.
• Port 42: This is the port number for NAMESERVER, which is a service that provides name resolution for the ARPANET. NAMESERVER was created in 1972 by Elizabeth Feinler, an American information scientist, who led the Network Information Center (NIC) at Stanford Research Institute. Feinler chose port 42 because it was the answer to the ultimate question of life, the universe, and everything, according to the novel The Hitchhiker’s Guide to the Galaxy.
• Port 22: This is the port number for SSH, which stands for Secure Shell. SSH is a protocol that allows you to securely access and control a remote computer, using encryption and authentication. SSH was created in 1995 by Tatu Ylönen, a Finnish researcher, who chose port 22 because it was his favorite number, and because it was close to port 23, which was used by Telnet, the insecure predecessor of SSH.

But most standard ports simply took the next available one, such as:

• Port 69: This is the port number for TFTP, which stands for Trivial File Transfer Protocol. It was not chosen because of the funny number. TFTP is a protocol that allows you to transfer files between computers, using a simple and minimalistic method. TFTP was created in 1980 by Noel Chiappa, a Canadian computer scientist, who was working on the MIT Multics project. Chiappa chose port 69 because it was the next available port number after port 68, which was used by BOOTP, a protocol that allows you to boot a computer from the network. Nice.
• Port 23: This is the port number for Telnet, which stands for Telecommunication Network. Telnet is a protocol that allows you to access and control a remote computer, using plain text commands and responses. Telnet was created in 1969, and it was one of the first protocols of the ARPANET. Telnet was assigned port 23 because it was the next available port number after port 22, which was used by TCPMUX at the time.
• Port 80: This is the port number for HTTP, which stands for Hypertext Transfer Protocol. HTTP is the protocol that allows you to access and view web pages, using a web browser and a web server. HTTP was created in 1989 by Tim Berners-Lee, the inventor of the World Wide Web. HTTP was assigned port 80 because it was the next available port number after port 79, which was used by FINGER at the time.
• Port 443: This is the port number for HTTPS, which stands for Hypertext Transfer Protocol Secure. HTTPS is the secure version of HTTP, which uses encryption and certificates to protect the data and the identity of the web server and the web browser. HTTPS was created in 1994 by Netscape, the company that developed the first popular web browser. HTTPS was assigned port 443 because it was the next available port number after port 442, which was used by SMSP, a protocol for sending and receiving short messages.

Guide to using the NGINX web server and reverse proxy

Nginx is a popular web server and reverse proxy that can handle high-performance and high-concurrency web applications. In this tutorial blog post, I will show you how to install and configure nginx on Ubuntu and RHEL-derivatives, how to use different conventions for managing nginx configuration files, and how to use

WirekatMarkus

The Mystery of Port Numbers

While most of the port numbers have a clear and logical explanation, some of them remain a mystery or a mystery. Some of them are assigned to services and protocols that are unknown, obsolete, or secretive. Some of them are used for malicious or illegal purposes, such as hacking, spamming, or botnetting. Some of them are just plain weird or funny. Here are some examples of the mystery of port numbers:

• Port 0: This is the port number that is reserved for the system, and it is not used by any service or protocol. It is sometimes used as a wildcard, to indicate any or all port numbers. It is also sometimes used as a source port, to indicate that the port number is irrelevant or unknown. Port 0 is also known as the null port, the reserved port, or the black hole port.
• Port 7: This is the port number for ECHO, which is a service that simply echoes back whatever data it receives. It is used for testing and debugging purposes, to check if the connection is working properly. However, port 7 is also used by some hackers and attackers, to create a denial-of-service attack, by sending a large amount of data to port 7, and causing the server to echo back the same amount of data, thus overwhelming the network bandwidth and resources.
• Port 19: This is the port number for CHARGEN, which is a service that generates a stream of random characters, such as letters, numbers, symbols, etc. It is used for testing and measuring the performance of the network, by sending and receiving large amounts of data. However, port 19 is also used by some hackers and attackers, to create a denial-of-service attack, by sending a large amount of data to port 19, and causing the server to generate the same amount of data, thus overwhelming the network bandwidth and resources.
• Port 80: This is the port number for HTTP, which is the protocol that allows you to access and view web pages, using a web browser and a web server. However, port 80 is also used by some hackers and attackers, to hide their malicious activities, by using port 80 as a disguise, and pretending to be a legitimate web service. For example, some malware and viruses use port 80 to communicate with their command and control servers, and to download or upload data, without being detected by firewalls or antivirus software.
• Port 31337: This is the port number for Back Orifice, which is a remote administration tool that allows you to access and control a remote computer, without the user’s knowledge or consent. It is used by some hackers and attackers, to spy on, steal, or damage the data and resources of the target computer. Back Orifice was created in 1998 by a group of hackers called the Cult of the Dead Cow, who chose port 31337 because it is the leet speak version of elite, and because it is a high and uncommon port number, that is unlikely to be used by other services or protocols.

The Future of Port Numbers

As the internet grows and evolves, so will the port numbers. New services and protocols will emerge and require new port numbers, and old services and protocols will fade and free up port numbers. The IANA will continue to assign and manage port numbers, and the internet community will continue to create and follow RFCs, to ensure the standardization and interoperability of port numbers.

As the number of services and protocols increases, the number of available port numbers decreases. This leads to port exhaustion. There are only 65536 port numbers, and more than half of them are already assigned or used. This means that there is a risk of running out of port numbers, or having port conflicts, where two or more services or protocols use the same port number. One possible solution to this problem is to use IPv6, which is the new version of the IP protocol, that allows for more IP addresses and more port numbers.

Conclusion

Port numbers are more than just numbers. They are the keys that unlock the doors of the internet. They are the codes that enable the communication and collaboration of the internet.

They are the stories that reveal the history and culture of the internet. Port numbers are the secret language of the internet.

While there unfortunately weren't that many interesting stories found about the origins of the port numbers, a few of them do tell a story.

Why uppercase SQL is so common, and why it doesn’t make sense

SQL, or Structured Query Language, is a widely used language for interacting with databases. SQL allows you to create, manipulate, and query data in a structured and efficient way. HOWEVER, THERE IS ONE ASPECT OF SQL THAT OFTEN SPARKS DEBATE AMONG DEVELOPERS - WHICH CASE DO YOU USE ..AND WHY

WirekatMarkus

What is WebAssembly? A New Standard for Web Development

WebAssembly, or Wasm for short, is a binary instruction format that can run on web browsers. It is designed to be fast, portable, and secure. Wasm aims to provide a common target for compiling high-level languages such as C, C++, Rust, and Go, and enable them to run on the

WirekatMarkus

SQL, or Structured Query Language, is a widely used language for interacting with databases. SQL allows you to create, manipulate, and query data in a structured and efficient way.

HOWEVER, THERE IS ONE ASPECT OF SQL THAT OFTEN SPARKS DEBATE AMONG DEVELOPERS - WHICH CASE DO YOU USE

Agenda

A few months ago I joined a project where a codebase was a mix of uppercase and lowercase SQL. I created a migration file and one comment during code review was "It would be better if this was uppercase, as it's good practice to do so". That gave me some mixed feelings - it's not like they followed any standard either.

Most of my previous teams and projects used lowercase SQL. Back when I studied at university or looked at SQL code samples online, a lot of times it's presented in uppercase. I started to wonder why. Is it actually "good practice"? Is it because we were just taught that way, or are there other reasons?

So I went on a research rabbithole. It all comes down to history.

Why Web3 is a Scam and a Fad

Web3 is the latest buzzword in the tech industry, promising to revolutionize the internet by creating a decentralized, peer-to-peer, and trustless network of applications and services. Web3 enthusiasts claim that this new paradigm will empower users, foster innovation, and challenge the dominance of big tech companies. Or so they claim.

WirekatMarkus

Everything you need to know about caching in Java & Spring Boot (plus Redis)

Caching is a technique of storing frequently used data in a fast and accessible memory, such as RAM, to reduce the latency and cost of retrieving it from a slower and more expensive source, such as a database or a network. Caching can improve the performance, scalability, and reliability of

WirekatMarkus

How uppercase SQL came to be

If you look at most SQL tutorials, books, or documentation, you will notice that the SQL keywords are usually written in uppercase.

This convention has a historical reason: back in the days when SQL was first developed, most computer systems used monospaced fonts and no syntax highlighting. This made it hard to distinguish between SQL keywords and other identifiers, such as table names, column names, or variables. To solve this problem, developers adopted the practice of writing SQL keywords in uppercase, so that they would stand out from the rest of the code.

Another major reason for using uppercase SQL keywords is that.. SQL is old.

SQL dates back to 1974. At that time, many keyboards only had uppercase letters, so the language documentation used them as well. This was a common practice in the early days of computing.

Stick to lowercase

However, times have changed, and so have the tools that we use to write and execute SQL. Nowadays, most modern editors and IDEs support proportional fonts and syntax highlighting, which make it much easier to read and write SQL code.

In fact, using uppercase SQL keywords can have some drawbacks, such as:

• It is harder to read: Studies have shown that lowercase text is easier to read than uppercase text, because lowercase letters have more distinctive shapes and patterns, while uppercase letters are more uniform and similar. This means that using uppercase SQL keywords can make your code less readable and more prone to errors.
• It is harder to type: Typing uppercase letters requires more keystrokes and more effort than typing lowercase letters, especially if you use a keyboard layout that requires you to press the Shift key or the Caps Lock key to switch between cases. This can slow down your typing speed and increase your fatigue. Rest that pinky!
• It is harder to maintain: If you use uppercase SQL keywords, you have to be consistent and use them throughout your code, otherwise your code will look messy and unprofessional. However, this can be tedious and error-prone, especially if you have to edit or refactor your code later. You may forget to change the case of some keywords, or introduce typos or inconsistencies. However, keep in mind that this goes both ways. It is, however, just easier to lowercase everything - let your editor handle the indentation and syntax highlightning for you.
• There’s no difference from a technical point of view. Your queries are the same speed in either case.

Many developers on Stackoverflow also do not endorse the use of uppercase syntax, and bring out some pretty great points.

Of course - if your codebase already has a set standard and actually enforces uppercase, then please do use it. I'm giving my pinky a break though.

11 Things You Can Do to Secure Your Linux Server

Linux is one of the most popular and widely used operating systems in the world, especially for servers. Linux servers power millions of websites, applications, databases, and other services that we use every day. However, Linux servers are not immune to cyberattacks, and they require proper security measures to protect

WirekatMarkus

The Bottom Line

In conclusion, the case of your SQL keywords doesn't really matter that much, because SQL is a case-insensitive language, meaning that it treats uppercase and lowercase letters as equivalent.

However, using lowercase SQL keywords can make your code more readable, more writable, and more maintainable, and improve your overall coding experience.

Therefore, I recommend that you use lowercase SQL keywords in your SQL code, and enjoy the benefits of this simple but effective practice.

Why do Linux users often prefer Thinkpads?

Readers like you help support Wirekat. If you purchase using links here, we may earn an affiliate commission. Read more If you are looking for a laptop that can run Linux flawlessly, you might want to consider a Thinkpad. Thinkpads are laptops that have been produced by IBM and Lenovo

WirekatMarkus

How a zero can change everything in Java (a puzzler)

Java is a popular and widely used programming language, but it also has some quirks and surprises that can puzzle even experienced developers. In this blog post, we will explore one such puzzler: what does the following code print? public class Puzzler { public static void main(String[] args) { int x

WirekatMarkus

What can you do on Linux that you can’t do on Windows?

Linux and Windows are two of the most popular operating systems in the world, but they have some significant differences in terms of features, flexibility, and usability. In this blog post, we will explore some of the things you can do on Linux that you can’t do on Windows,

WirekatMarkus

Beginner’s guide to the Linux terminal. Everything you need to know in under 8 minutes of your time

Linux is an open-source operating system that powers many servers, supercomputers, and devices. Learning Linux can boost your career as a software developer, system administrator, cybersecurity professional, or IT support specialist. In this blog post, I will introduce one of the most useful skills you need to master in Linux:

WirekatMarkus

The Single Responsibility Principle (SRP) is one of the most important concepts in software design. It states that every class or module should have one and only one reason to change. In other words, each class or module should have a single responsibility or a single purpose within the system.

The benefits of applying the SRP are:

• It makes the code more cohesive, meaning that the elements of a class or module are closely related and work together towards a common goal.
• It makes the code more decoupled, meaning that the classes or modules have minimal dependencies on each other and can be easily reused or modified without affecting other parts of the system.
• It makes the code more readable, meaning that the classes or modules have clear and meaningful names and functions that reflect their responsibility.
• It makes the code more testable, meaning that the classes or modules can be easily isolated and verified with unit tests.

To illustrate the SRP, let’s look at some examples of bad and good design in Kotlin code. The same should apply to any other language, eg Java.

Example 1

Bad Example

Suppose we have a class called User that represents a user of an online shopping system. The class has the following properties and functions:

class User(
    val id: Int,
    val name: String,
    val email: String,
    val address: String,
    val cart: MutableList<Product>
) {
    fun addToCart(product: Product) {
        cart.add(product)
    }

    fun removeFromCart(product: Product) {
        cart.remove(product)
    }

    fun checkout() {
        // calculate the total price of the cart
        var total = 0.0
        for (product in cart) {
            total += product.price
        }
        // apply a discount if applicable
        if (cart.size >= 10) {
            total *= 0.9
        }
        // send an email confirmation to the user
        val message = "Dear $name, \n" +
                "Thank you for your purchase. \n" +
                "Your order total is $$total. \n" +
                "Your order will be shipped to $address. \n" +
                "Please reply to this email if you have any questions or concerns."
        sendEmail(email, message)
    }

    fun sendEmail(to: String, message: String) {
        // create an email object
        val email = Email(to, "Order Confirmation", message)
        // connect to an SMTP server
        val smtp = SmtpClient("smtp.example.com", 25)
        // send the email
        smtp.send(email)
    }
}

This class violates the SRP because it has more than one reason to change. It has multiple responsibilities, such as:

• Managing the user’s personal information (id, name, email, address)
• Managing the user’s shopping cart (cart, addToCart, removeFromCart)
• Performing the checkout logic (checkout, sendEmail)

If any of these responsibilities change, the class will have to be modified, which increases the risk of introducing bugs and breaking other functionalities. For example, if the business rules for the checkout logic change, such as adding a new discount policy or a new payment method, the User class will have to be updated. This could affect the other functions of the class, such as managing the user’s information or shopping cart.

Good Example

To apply the SRP, we should split the User class into smaller classes that have a single responsibility each. For example, we could have the following classes:

// A class that represents a user's personal information
class User(
    val id: Int,
    val name: String,
    val email: String,
    val address: String
)

// A class that represents a user's shopping cart
class Cart(
    val user: User,
    val items: MutableList<Product>
) {
    fun addToCart(product: Product) {
        items.add(product)
    }

    fun removeFromCart(product: Product) {
        items.remove(product)
    }
}

// A class that performs the checkout logic
class CheckoutService {
    fun checkout(cart: Cart) {
        // calculate the total price of the cart
        var total = 0.0
        for (product in cart.items) {
            total += product.price
        }
        // apply a discount if applicable
        if (cart.items.size >= 10) {
            total *= 0.9
        }
        // send an email confirmation to the user
        val message = "Dear ${cart.user.name}, \n" +
                "Thank you for your purchase. \n" +
                "Your order total is $$total. \n" +
                "Your order will be shipped to ${cart.user.address}. \n" +
                "Please reply to this email if you have any questions or concerns."
        EmailService.sendEmail(cart.user.email, message)
    }
}

// A class that handles the email communication
class EmailService {
    companion object {
        fun sendEmail(to: String, message: String) {
            // create an email object
            val email = Email(to, "Order Confirmation", message)
            // connect to an SMTP server
            val smtp = SmtpClient("smtp.example.com", 25)
            // send the email
            smtp.send(email)
        }
    }
}

This design follows the SRP because each class has a single responsibility and a single reason to change. The User class only manages the user’s personal information, the Cart class only manages the user’s shopping cart, the CheckoutService class only performs the checkout logic, and the EmailService class only handles the email communication. If any of these responsibilities change, only the corresponding class will have to be modified, which reduces the coupling and increases the cohesion of the code.

Example 2

Bad Example

Suppose we have a class called Book that represents a book with some properties and functions:

class Book(
    val title: String,
    val author: String,
    val content: String
) {
    fun print() {
        // print the book content to a printer
    }

    fun format() {
        // format the book content according to some rules
    }

    fun publish() {
        // upload the book content to a publishing platform
    }
}

This class violates the SRP because it has more than one reason to change. It has multiple responsibilities, such as:

• Managing the book information (title, author, content)
• Printing the book content (print)
• Formatting the book content (format)
• Publishing the book content (publish)

If any of these responsibilities change, the class will have to be modified, which increases the risk of introducing bugs and breaking other functionalities. For example, if the printing or publishing platform changes, the Book class will have to be updated. This could affect the other functions of the class, such as managing the book information or formatting the book content.

Good Example

To apply the SRP, we should split the Book class into smaller classes that have a single responsibility each. For example, we could have the following classes:

// A class that represents a book's information
class Book(
    val title: String,
    val author: String,
    val content: String
)

// A class that prints the book content
class BookPrinter {
    fun print(book: Book) {
        // print the book content to a printer
    }
}

// A class that formats the book content
class BookFormatter {
    fun format(book: Book) {
        // format the book content according to some rules
    }
}

// A class that publishes the book content
class BookPublisher {
    fun publish(book: Book) {
        // upload the book content to a publishing platform
    }
}

This design follows the SRP because each class has a single responsibility and a single reason to change. The Book class only manages the book information, the BookPrinter class only prints the book content, the BookFormatter class only formats the book content, and the BookPublisher class only publishes the book content. If any of these responsibilities change, only the corresponding class will have to be modified, which reduces the coupling and increases the cohesion of the code.

Example 3

Bad Example

Suppose we have a class called Calculator that represents a calculator application with some properties and functions:

class Calculator(
    var input: String,
    var output: String
) {
    fun parseInput() {
        // parse the input string into numbers and operators
    }

    fun performOperation() {
        // perform the arithmetic operation based on the input
    }

    fun displayOutput() {
        // display the output string on the screen
    }

    fun saveOutput() {
        // save the output string to a file
    }
}

This class violates the SRP because it has more than one reason to change. It has multiple responsibilities, such as:

• Managing the input and output strings (input, output)
• Parsing the input string into numbers and operators (parseInput)
• Performing the arithmetic operation based on the input (performOperation)
• Displaying the output string on the screen (displayOutput)
• Saving the output string to a file (saveOutput)

If any of these responsibilities change, the class will have to be modified, which increases the risk of introducing bugs and breaking other functionalities. For example, if the user interface or the file format changes, the Calculator class will have to be updated. This could affect the other functions of the class, such as parsing the input or performing the operation.

Good Example

To apply the SRP, we should split the Calculator class into smaller classes that have a single responsibility each. For example, we could have the following classes:

// A class that represents the input and output strings
class CalculatorData(
    var input: String,
    var output: String
)

// A class that parses the input string into numbers and operators
class CalculatorParser {
    fun parseInput(data: CalculatorData) {
        // parse the input string into numbers and operators
    }
}

// A class that performs the arithmetic operation based on the input
class CalculatorOperation {
    fun performOperation(data: CalculatorData) {
        // perform the arithmetic operation based on the input
    }
}

// A class that displays the output string on the screen
class CalculatorDisplay {
    fun displayOutput(data: CalculatorData) {
        // display the output string on the screen
    }
}

// A class that saves the output string to a file
class CalculatorStorage {
    fun saveOutput(data: CalculatorData) {
        // save the output string to a file
    }
}

This design follows the SRP because each class has a single responsibility and a single reason to change. The CalculatorData class only manages the input and output strings, the CalculatorParser class only parses the input string into numbers and operators, the CalculatorOperation class only performs the arithmetic operation based on the input, the CalculatorDisplay class only displays the output string on the screen, and the CalculatorStorage class only saves the output string to a file. If any of these responsibilities change, only the corresponding class will have to be modified, which reduces the coupling and increases the cohesion of the code.

Conclusion

The Single Responsibility Principle is a key principle of software design that helps to create more maintainable, readable, and testable code. By following the SRP, we can avoid the problems of having classes or modules that are too large, complex, or dependent on each other, and instead have smaller, simpler, and more independent units of code that are easier to work with.

Logging is an essential part of any software development process. Logging is the act of recording events or messages that occur during the execution of a program. Logging can help developers to debug, monitor, and troubleshoot their applications, as well as provide valuable insights into the performance, behavior, and usage of their systems.

However, logging is not as simple as printing some text to the console or a file. Logging requires careful planning, design, and implementation to ensure that it is effective, efficient, and consistent. In this article, we will discuss some of the best practices for logging in Spring Boot applications, as well as provide a comprehensive tutorial on how to use the popular logging framework Logback with Spring Boot.

Everything you need to know about caching in Java & Spring Boot (plus Redis)

Caching is a technique of storing frequently used data in a fast and accessible memory, such as RAM, to reduce the latency and cost of retrieving it from a slower and more expensive source, such as a database or a network. Caching can improve the performance, scalability, and reliability of

WirekatMarkus

How to use the ELK stack with Spring Boot

The ELK stack is a popular combination of open-source tools for collecting, storing, analyzing, and visualizing data. ELK stands for Elasticsearch, Logstash, and Kibana. Elasticsearch is a distributed search and analytics engine that can handle large volumes of structured and unstructured data. Logstash is a data processing pipeline that can

WirekatMarkus

Best Practices for Logging

Logging is not a one-size-fits-all solution. Different applications may have different logging requirements and preferences. However, there are some general principles and guidelines that can help developers to achieve better logging quality and consistency. Here are some of the best practices for logging:

• Use a logging framework. A logging framework is a library or tool that provides a standardized and configurable way of logging messages. A logging framework can handle various aspects of logging, such as formatting, filtering, routing, and storing the log messages. A logging framework can also provide features such as log levels, appenders, layouts, and patterns. Using a logging framework can make logging easier, more flexible, and more consistent across different modules and environments. Some of the popular logging frameworks for Java are Logback, Log4j, and SLF4J.
• Use appropriate log levels. Log levels are a way of categorizing the log messages according to their severity or importance. Log levels can help developers to control the amount and detail of the log output, as well as filter and prioritize the log messages. Log levels can also help users and operators to identify and troubleshoot issues more quickly and effectively. The common log levels are:
  • TRACE: The most detailed and verbose level of logging. TRACE messages are usually used for debugging purposes, to track the flow and state of the program. TRACE messages should only be enabled in development or testing environments, as they can generate a lot of noise and overhead in production.
  • DEBUG: A less detailed level of logging than TRACE. DEBUG messages are also used for debugging purposes, to provide more information about the program logic and behavior. DEBUG messages can be useful for developers to diagnose and fix problems, but they should also be disabled in production, as they can still produce a lot of output and affect performance.
  • INFO: A general level of logging that provides informative and useful messages about the program. INFO messages are used to report the normal and expected events and operations of the program, such as startup, shutdown, configuration, status, etc. INFO messages can be helpful for users and operators to monitor and understand the program, and they should be enabled in production, unless they are too frequent or redundant.
  • WARN: A higher level of logging that indicates a potential problem or issue. WARN messages are used to report the abnormal or unexpected events and conditions of the program, such as errors, failures, violations, etc. WARN messages can alert users and operators to take preventive or corrective actions, and they should always be enabled in production, unless they are false positives or irrelevant.
  • ERROR: The highest level of logging that indicates a serious problem or issue. ERROR messages are used to report the critical and fatal errors and exceptions of the program, such as crashes, data loss, corruption, etc. ERROR messages can indicate that the program is unable to function properly or at all, and they should always be enabled in production, as they require immediate attention and resolution.
• Use descriptive and consistent messages. The content and format of the log messages are also important for logging quality and consistency. Log messages should be descriptive and informative, providing enough detail and context to understand the event or issue. Log messages should also be consistent and follow a standard pattern or convention, such as using the same date and time format, delimiter, prefix, suffix, etc. Log messages should also avoid using ambiguous or vague terms, such as “error”, “failed”, “success”, etc., without specifying the cause, effect, or action. Log messages should also avoid using sensitive or confidential information, such as passwords, tokens, personal data, etc., as they can pose security and privacy risks.
• Use structured and machine-readable formats. The format and structure of the log messages are also important for logging efficiency and usability. Log messages should be structured and machine-readable, using a common and well-defined format, such as JSON, XML, CSV, etc. Structured and machine-readable log messages can make logging more scalable, reliable, and interoperable, as they can be easily parsed, processed, and transmitted by various tools and systems. Structured and machine-readable log messages can also make logging more searchable, analyzable, and actionable, as they can be easily queried, filtered, aggregated, and visualized by various tools and platforms.
• Use appropriate appenders and destinations. The appender and destination of the log messages are also important for logging performance and availability. The appender and destination are the components or mechanisms that handle the output and storage of the log messages. The appender and destination can vary depending on the type, format, and location of the log output, such as console, file, database, network, etc. The appender and destination should be appropriate and suitable for the logging requirements and preferences of the application, such as the volume, frequency, and retention of the log messages. The appender and destination should also be reliable and resilient, ensuring that the log messages are not lost, corrupted, or delayed.

How a zero can change everything in Java (a puzzler)

Java is a popular and widely used programming language, but it also has some quirks and surprises that can puzzle even experienced developers. In this blog post, we will explore one such puzzler: what does the following code print? public class Puzzler { public static void main(String[] args) { int x

WirekatMarkus

The default logging behavior in Spring Boot

Before we customize and configure the logging behavior of our application, it is useful to understand the default logging behavior that Spring Boot provides. Spring Boot uses a sensible and opinionated logging configuration that works well for most applications. The default logging behavior includes the following aspects:

• The default logging framework is Logback, which is a successor of Log4j and provides a simple and powerful way of logging messages.
• The default logging destination is the console, which means that the log messages are printed to the standard output stream (System.out) or the standard error stream (System.err) depending on the log level.
• The default logging file or path is not set, which means that the log messages are not written to any file or folder. However, we can set the logging.file.name or logging.file.path properties to enable file output, as will be explained below.
• The default logging format is color-coded and structured, which means that the log messages are displayed and organized using various elements and symbols, such as date, time, level, logger, message, etc. The log messages also use different colors to indicate the log level, such as red for ERROR, yellow for WARN, green for INFO, etc.
• The default logging pattern is “%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n”, which means that the log messages follow a standard template that consists of the following elements:
  • %d{yyyy-MM-dd HH:mm:ss.SSS}: The date and time of the log message, in the format yyyy-MM-dd HH:mm:ss.SSS, such as 2024-02-10 11:59:13.123
  • [%thread]: The thread name of the log message, in square brackets […], such as [main]
  • %-5level: The log level of the log message, in uppercase letters, padded with spaces to 5 characters, such as INFO , DEBUG, WARN
  • %logger{36}: The logger name of the log message, truncated to 36 characters, such as com.example.App, org.springframework.boot.web.embedded.tomcat.TomcatWebServer
    • : A dash symbol - to separate the elements
  • %msg: The log message itself, which can be any text or object, such as Application started successfully
  • %n: A newline character \n to end the line
• The default logging level is INFO, which means that the log messages with the INFO level or higher (WARN and ERROR) are outputted, while the log messages with the lower levels (DEBUG and TRACE) are ignored. However, we can change the logging level by using the logging.level property or the --debug or --trace arguments, as we saw in the previous steps.
• The default logging rotation and compression are not enabled, which means that the log messages are not archived or compressed after a certain time or size. However, we can enable logging rotation and compression by using the logging.file.max-size, logging.file.max-history, and logging.file.total-size-cap properties, as we will see in the next steps.

Tutorial on Logging with Logback and Spring Boot

In this tutorial, we will demonstrate how to use the logging framework Logback with Spring Boot, a popular framework for building Java applications. Logback is a successor of Log4j, and it is the default logging framework for Spring Boot. Logback provides a simple and powerful way of logging messages, using various components such as loggers, appenders, layouts, and filters. Logback also supports various features such as log levels, patterns, variables, and profiles.

To follow this tutorial, you will need:

• Java 11 or higher
• Maven 3.6 or higher
• Spring Boot 2.5 or higher

Step 1: Create a Spring Boot project

The first step is to create a Spring Boot project using Maven. You can use the Spring Initializr website to generate the project.

Next, we need to add the Spring Boot dependencies and plugins to the pom.xml file. We will also add the Logback Classic dependency, which is the core module of Logback that provides the logging functionality. 

        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-classic</artifactId>
            <version>1.2.6</version>
        </dependency>

Step 2: Create a Spring Boot application class

The next step is to create a Spring Boot application class that will run the application and initialize the logging system. We will use the @SpringBootApplication annotation to mark the class as a Spring Boot application, and the SpringApplication.run() method to launch the application. We will also use the LoggerFactory class to obtain a logger instance for the class, and the log.info() method to log a message at the INFO level. The application class should look like this:

package com.example;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class App {

    private static final Logger log = LoggerFactory.getLogger(App.class);

    public static void main(String[] args) {
        SpringApplication.run(App.class, args);
        log.info("Application started successfully");
    }
}

Alternatively if you use Lombok, you can append a @Slf4j annotation and use log.* the same way.

The application will log various messages at different levels, such as INFO, DEBUG, and WARN. The log messages also follow a standard pattern, which consists of the following elements:

• The timestamp of the log message, in the format HH:mm:ss.SSS
• The thread name of the log message, in square brackets [...]
• The log level of the log message, in uppercase letters INFO, DEBUG, WARN, etc.
• The logger name of the log message, which is usually the fully qualified class name com.example.App, org.springframework.boot.web.embedded.tomcat.TomcatWebServer, etc.
• The log message itself, which can be any text or object Starting App using Java 11.0.12 on copilot.local with PID 12345, Tomcat started on port(s): 8080 (http) with context path '', etc.

This is the default logging configuration and behavior of Spring Boot and Logback. However, we can customize and configure various aspects of logging, such as the log level, the log pattern, the log destination, etc. We will see how to do that in the next steps.

The 10 Hardest Concepts in Java (With code examples)

Java is one of the most popular and widely used programming languages in the world. It is known for its simplicity, portability, and versatility. However, Java is not without its challenges. There are some concepts that are harder to understand and master than others, and require more time, effort, and

WirekatMarkus

Step 3: Control log levels in the application.yml file

One of the ways to control the log levels of the application is to use the application.yml file, which is a configuration file that Spring Boot uses to load various properties and settings. The application.yml file is located in the src/main/resources folder of the project, and it can be created if it does not exist.

To control the log levels of the application, we can use the logging.level property, which allows us to specify the log level for a specific logger or a package. The syntax of the property is:

logging.level.<logger-name-or-package>: <log-level>

For example, if we want to set the log level of the root logger (which is the parent of all other loggers) to WARN, we can use the following property:

logging.level.root: WARN

If we want to set the log level of the com.example package (which is the base package of our application) to DEBUG, we can use the following property:

logging.level.com.example: DEBUG

If we want to set the log level of the org.springframework package (which is the base package of the Spring framework) to INFO, we can use the following property:

logging.level.org.springframework: INFO

We can also use the logging.level.* property to set the log level for all other loggers that are not explicitly specified. For example, if we want to set the default log level to ERROR, we can use the following property:

logging.level.*: ERROR

The log levels that we can use are the same as the ones we discussed in the previous section: TRACE, DEBUG, INFO, WARN, and ERROR. The log levels are hierarchical, meaning that a higher level includes all the lower levels. For example, if we set the log level to WARN, it will also include ERROR messages, but not INFO, DEBUG, or TRACE messages.

The log levels that we specify in the application.yml file will override the default log levels that are configured by Spring Boot and Logback. However, we can also use the spring.profiles.active property to activate different log levels for different profiles or environments. For example, if we want to use different log levels for development and production profiles, we can use the following properties:

spring.profiles.active: dev

---

spring.profiles: dev
logging.level.root: DEBUG

---

spring.profiles: prod
logging.level.root: WARN

The --- symbol is used to separate different sections or documents in the application.yml file. The spring.profiles.active property specifies the active profile for the application, which is dev in this case. The spring.profiles property specifies the profile for each section, which can be dev or prod in this case. The logging.level.root property specifies the log level for the root logger for each profile, which is DEBUG for dev and WARN for prod in this case.

To run the application with a specific profile, we can use the following command:

mvn spring-boot:run -Dspring-boot.run.profiles=prod

This will run the application with the prod profile, which will use the log level of WARN for the root logger.

Step 4: Configure logging format in the application.yml file

Another way to customize the logging configuration of the application is to use the application.yml file to configure the logging format, which is the way the log messages are displayed and structured. The logging format can affect the readability, usability, and interoperability of the log messages.

To configure the logging format of the application, we can use the logging.pattern property, which allows us to specify the pattern or template for the log messages. The syntax of the property is:

logging.pattern.<console-or-file>: <pattern>

The <console-or-file> parameter specifies the destination of the log messages, which can be either console or file. The <pattern> parameter specifies the pattern or template for the log messages, which can be a combination of various elements or tokens, such as date, time, level, logger, message, etc. The pattern can also include various symbols or characters, such as spaces, commas, brackets, etc.

For example, if we want to change the pattern for the console output, we can use the following property:

logging.pattern.console: "%d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"

This will change the pattern for the console output to include the following elements:

• %d{yyyy-MM-dd HH:mm:ss.SSS}: The date and time of the log message, in the format yyyy-MM-dd HH:mm:ss.SSS, such as 2024-02-10 11:59:13.123
• [%t]: The thread name of the log message, in square brackets [...], such as [main]
• %-5level: The log level of the log message, in uppercase letters, padded with spaces to 5 characters, such as INFO , DEBUG, WARN
• %logger{36}: The logger name of the log message, truncated to 36 characters, such as com.example.App, org.springframework.boot.web.embedded.tomcat.TomcatWebServer
• -: A dash symbol - to separate the elements
• %msg: The log message itself, which can be any text or object, such as Application started successfully
• %n: A newline character \n to end the line

The result of this pattern will look like this:

2024-02-10 11:59:13.123 [main] INFO  com.example.App - Application started successfully

If we want to change the pattern for the file output, we can use the following property:

logging.pattern.file: "%d{ISO8601} [%t] %-5level %logger{36} - %msg%n"

This will change the pattern for the file output to include the following elements:

• %d{ISO8601}: The date and time of the log message, in the ISO 8601 format, such as 2024-02-10T11:59:13.123+00:00
• [%t]: The thread name of the log message, in square brackets [...], such as [main]
• %-5level: The log level of the log message, in uppercase letters, padded with spaces to 5 characters, such as INFO , DEBUG, WARN
• %logger{36}: The logger name of the log message, truncated to 36 characters, such as com.example.App, org.springframework.boot.web.embedded.tomcat.TomcatWebServer
• -: A dash symbol - to separate the elements
• %msg: The log message itself, which can be any text or object, such as Application started successfully
• %n: A newline character \n to end the line

The result of this pattern will look like this:

2024-02-10T11:59:13.123+00:00 [main] INFO  com.example.App - Application started successfully

The pattern elements that we can use are defined by the Logback framework, and they can be found in the Logback documentation. We can also use custom or user-defined elements, such as variables, colors, markers, etc., by using the %X, %highlight, %marker, etc. tokens.

The logging format that we specify in the application.yml file will override the default logging format that is configured by Spring Boot and Logback. However, we can also use the spring.profiles.active property to activate different logging formats for different profiles or environments. For example, if we want to use different logging formats for development and production profiles, we can use the following properties:

spring.profiles.active: dev

---

spring.profiles: dev
logging.pattern.console: "%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} - %msg%n"

---

spring.profiles: prod
logging.pattern.console: "%d{ISO8601} [%t] %-5level %logger{36} - %msg%n"

The --- symbol is used to separate different sections or documents in the application.yml file. The spring.profiles.active property specifies the active profile for the application, which is dev in this case. The spring.profiles property specifies the profile for each section, which can be dev or prod in this case. The logging.pattern.console property specifies the pattern for the console output for each profile, which is HH:mm:ss.SSS for dev and ISO8601 for prod in this case.

To run the application with a specific profile, we can use the following command:

mvn spring-boot:run -Dspring-boot.run.profiles=prod

This will run the application with the prod profile, which will use the ISO 8601 format for the console output.

Step 5: Configure logging destination in the application.yml file

Another aspect of logging configuration that we can customize is the logging destination, which is the place or medium where the log messages are outputted and stored. The logging destination can affect the performance, availability, and security of the log messages.

To configure the logging destination of the application, we can use the logging.file or logging.path properties, which allow us to specify the file or folder for the log output. The syntax of the properties is:

logging.file.name: <file-name>
logging.file.path: <folder-path>

The <file-name> parameter specifies the name of the file for the log output, which can be any valid file name, such as application.log, myapp.log, etc. The <folder-path> parameter specifies the path of the folder for the log output, which can be any valid folder path, such as /var/log, /tmp, etc.

For example, if we want to output the log messages to a file named application.log in the current working directory, we can use the following property:

logging.file.name: application.log

If we want to output the log messages to a file named myapp.log in the /var/log folder, we can use the following properties:

logging.file.name: myapp.log
logging.file.path: /var/log

The logging file or path that we specify in the application.yml file will override the default logging destination that is configured by Spring Boot and Logback. However, we can also use the spring.profiles.active property to activate different logging destinations for different profiles or environments. For example, if we want to use different logging destinations for development and production profiles, we can use the following properties:

spring.profiles.active: dev

---

spring.profiles: dev
logging.file.name: application-dev.log

---

spring.profiles: prod
logging.file.name: application-prod.log
logging.file.path: /var/log

The --- symbol is used to separate different sections or documents in the application.yml file. The spring.profiles.active property specifies the active profile for the application, which is dev in this case. The spring.profiles property specifies the profile for each section, which can be dev or prod in this case. The logging.file.name and logging.file.path properties specify the file name and folder path for the log output for each profile, which are application-dev.log and application-prod.log in the current working directory and /var/log folder respectively in this case.

To run the application with a specific profile, we can use the following command:

mvn spring-boot:run -Dspring-boot.run.profiles=prod

This will run the application with the prod profile, which will output the log messages to the file application-prod.log in the /var/log folder.

Step 6: Enable logging rotation and compression in the application.yml file

Another aspect of logging configuration that we can customize is the logging rotation and compression, which are the processes of archiving and compressing the log files after a certain time or size. The logging rotation and compression can help to manage the disk space and performance of the log files, as well as to improve the security and retention of the log messages.

To enable logging rotation and compression of the application, we can use the logging.file.max-size, logging.file.max-history, and logging.file.total-size-cap properties, which allow us to specify the size, history, and cap of the log files. The syntax of the properties is:

logging.file.max-size: <size>
logging.file.max-history: <number>
logging.file.total-size-cap: <size>

The <size> parameter specifies the maximum size of the log file or the total size of the log files, which can be any valid size unit, such as 10MB, 100KB, 1GB, etc. The <number> parameter specifies the maximum number of the log files to keep, which can be any positive integer, such as 7, 30, 365, etc.

For example, if we want to rotate the log file every 10 MB, and keep up to 7 log files, we can use the following properties:

logging.file.max-size: 10MB
logging.file.max-history: 7

This will rotate the log file every 10 MB, and keep up to 7 log files, such as application.log, application.log.1, application.log.2, …, application.log.7. The oldest log file will be deleted when a new log file is created.

If we want to compress the log files after rotation, we can use the .gz or .zip extension for the log file name, such as:

logging.file.name: application.log.gz

This will compress the log files after rotation, and keep up to 7 compressed log files, such as application.log.gz, application.log.1.gz, application.log.2.gz, …, application.log.7.gz. The compressed log files will take less disk space than the uncompressed log files.

If we want to limit the total size of the log files, we can use the logging.file.total-size-cap property, such as:

logging.file.total-size-cap: 100MB

This will limit the total size of the log files to 100 MB, and delete the oldest log files when the cap is reached.

The logging rotation and compression properties that we specify in the application.yml file will override the default logging rotation and compression behavior that is configured by Spring Boot and Logback. However, we can also use the spring.profiles.active property to activate different logging rotation and compression settings for different profiles or environments. For example, if we want to use different logging rotation and compression settings for development and production profiles, we can use the following properties:

spring.profiles.active: dev

---

spring.profiles: dev
logging.file.name: application-dev.log
logging.file.max-size: 10MB
logging.file.max-history: 7

---

spring.profiles: prod
logging.file.name: application-prod.log.gz
logging.file.max-size: 10MB
logging.file.max-history: 30
logging.file.total-size-cap: 1GB

The --- symbol is used to separate different sections or documents in the application.yml file. The spring.profiles.active property specifies the active profile for the application, which is dev in this case. The spring.profiles property specifies the profile for each section, which can be dev or prod in this case. The logging.file.name, logging.file.max-size, logging.file.max-history, and logging.file.total-size-cap properties specify the file name, size, history, and cap for the log files for each profile, which are different for dev and prod in this case.

To run the application with a specific profile, we can use the following command:

mvn spring-boot:run -Dspring-boot.run.profiles=prod

This will run the application with the prod profile, which will rotate and compress the log files every 10 MB, and keep up to 30 log files with a total size cap of 1 GB.

Example: Compress logs after one day

To configure Spring Boot to compress the logs after one day, you can use the logging.file.max-history property in the application.yml file, and set it to 1. This will keep only one day of log files, and compress the older ones with the .gz extension. For example:

logging.file.name: application.log.gz
logging.file.max-history: 1

Example: Compress logs after a month

To configure Spring Boot to compress the logs after one month, you can use the same property, but set it to 30. This will keep 30 days of log files, and compress the older ones with the .gz extension. For example:

logging.file.name: application.log.gz
logging.file.max-history: 30

You can also use the logging.file.max-size property to limit the size of each log file, and the logging.file.total-size-cap property to limit the total size of all log files. 

Step 7: Follow the best practices on storing logs

Storing logs is an important part of log management, as it affects the availability, security, and compliance of the log data. Storing logs requires careful planning and implementation, as it involves various factors such as the location, format, size, and retention of the log files. Here are some of the best practices on storing logs:

• Choose a suitable location for your log files. The location of your log files can have a significant impact on the performance, reliability, and security of your logging system. You should choose a location that is accessible, scalable, and resilient, such as a dedicated disk, a network share, a cloud storage, or a log management service. You should also avoid storing your log files on the same device or server as your application, as it can cause performance degradation, resource contention, and security risks. You should also consider the network bandwidth, latency, and cost of transferring your log files to the desired location.
• Use a standard and structured format for your log files. The format of your log files can affect the readability, usability, and interoperability of your log data. You should use a standard and structured format for your log files, such as JSON, XML, CSV, etc. A standard and structured format can make your log files more consistent, parsable, and searchable, as well as more compatible with various tools and systems. You should also use a common and well-defined naming convention for your log files, such as using the date, time, application name, or log level as part of the file name.
• Optimize the size of your log files. The size of your log files can affect the storage capacity, performance, and cost of your logging system. You should optimize the size of your log files by using appropriate log levels, filtering, sampling, and compression techniques. You should use the lowest log level that meets your logging objectives, and filter out any unnecessary or redundant log messages. You should also sample your log messages to reduce the volume and frequency of the log output, and compress your log files to save disk space and bandwidth. You should also use a suitable unit for the size of your log files, such as KB, MB, GB, etc.
• Establish a log retention policy. A log retention policy is a set of rules that defines how long and how much log data you should keep. A log retention policy can help you balance the trade-off between the value and the cost of your log data, as well as comply with any legal or regulatory requirements. You should establish a log retention policy that is aligned with your logging objectives, business needs, and compliance obligations. You should also consider the storage capacity, performance, and cost of your logging system, as well as the relevance, usefulness, and sensitivity of your log data. You should also review and update your log retention policy regularly, and enforce it consistently.

Step 8: Enable automatic log deletion in Spring Boot

One of the ways to implement a log retention policy in Spring Boot is to enable automatic log deletion, which is the process of removing old or obsolete log files after a certain time or size. Automatic log deletion can help you manage the disk space and performance of your log files, as well as improve the security and compliance of your log data.

To enable automatic log deletion in Spring Boot, you can use the logging.file.max-history and logging.file.total-size-cap properties, which allow you to specify the history and cap of the log files. The syntax of the properties is:

logging.file.max-history: <number>
logging.file.total-size-cap: <size>

The <number> parameter specifies the maximum number of the log files to keep, which can be any positive integer, such as 7, 30, 365, etc. The <size> parameter specifies the maximum total size of the log files to keep, which can be any valid size unit, such as 10MB, 100KB, 1GB, etc.

For example, if we want to delete the log files that are older than 30 days, we can use the following property:

logging.file.max-history: 30

This will delete the log files that are older than 30 days, and keep up to 30 log files.

If we want to delete the log files that exceed 1 GB in total size, we can use the following property:

logging.file.total-size-cap: 1GB

This will delete the oldest log files when the total size of the log files exceeds 1 GB, and keep the log files within the 1 GB cap.

The automatic log deletion properties that we specify in the application.yml file will override the default log deletion behavior that is configured by Spring Boot and Logback. However, we can also use the spring.profiles.active property to activate different log deletion settings for different profiles or environments. For example, if we want to use different log deletion settings for development and production profiles, we can use the following properties:

spring.profiles.active: dev

---

spring.profiles: dev
logging.file.max-history: 7
logging.file.total-size-cap: 100MB

---

spring.profiles: prod
logging.file.max-history: 30
logging.file.total-size-cap: 1GB

The --- symbol is used to separate different sections or documents in the application.yml file. The spring.profiles.active property specifies the active profile for the application, which is dev in this case. The spring.profiles property specifies the profile for each section, which can be dev or prod in this case. The logging.file.max-history and logging.file.total-size-cap properties specify the history and cap for the log files for each profile, which are different for dev and prod in this case.

To run the application with a specific profile, we can use the following command:

mvn spring-boot:run -Dspring-boot.run.profiles=prod

This will run the application with the prod profile, which will delete the log files that are older than 30 days or exceed 1 GB in total size.

I want to store logs for a longer duration

If you want to delete logs after let's say, 6 months, you can use the logging.file.max-history property in the application.yml file, and set it to 180. This will keep 180 days of log files, and delete the older ones. For example:

logging.file.name: application.log.gz
logging.file.max-history: 180

Conclusion

In this article, we have discussed some of the best practices and guidelines for logging in Spring Boot applications, such as using a logging framework, using appropriate log levels, using descriptive and consistent messages, using structured and machine-readable formats, and using appropriate appenders and destinations.

We have also provided a comprehensive tutorial on how to use the logging framework Logback with Spring Boot, and how to customize and configure various aspects of logging, such as the log level, the log pattern, the log destination, etc. We have also shown how to use the application.yml file to control the logging configuration for different profiles or environments.

Logging is a vital part of any software development process, and it can help developers to improve the quality, reliability, and security of their applications. Logging can also help users and operators to monitor, understand, and troubleshoot their systems. Logging requires careful planning, design, and implementation to ensure that it is effective, efficient, and consistent. We hope that this article has provided some useful information and tips on how to log better in Spring Boot.

With all the logging in place, you might consider setting up a monitoring and alerting solution, as the one demonstrated below:

How to use the ELK stack with Spring Boot

The ELK stack is a popular combination of open-source tools for collecting, storing, analyzing, and visualizing data. ELK stands for Elasticsearch, Logstash, and Kibana. Elasticsearch is a distributed search and analytics engine that can handle large volumes of structured and unstructured data. Logstash is a data processing pipeline that can

WirekatMarkus

JavaScript is a language that allows you to declare variables in different ways. You can use the var keyword, which creates a variable that is function-scoped. This means that the variable is accessible within the function where it is declared, and any nested functions. For example:

function foo() {
  var x = 1; // this is a function-scoped variable
  function bar() {
    console.log(x); // this will print 1, because x is accessible here
  }
  bar();
}
foo();

You can also use the let or const keywords, which create variables that are block-scoped. This means that the variable is accessible within the block where it is declared, and any nested blocks. A block is a set of statements enclosed by curly braces, such as an if statement, a for loop, or a try block. For example:

function foo() {
  if (true) {
    let x = 1; // this is a block-scoped variable
    console.log(x); // this will print 1, because x is accessible here
  }
  console.log(x); // this will throw an error, because x is not accessible here
}
foo();

The unexpected results of a missing comma in JavaScript

JavaScript is a language full of surprises. Sometimes, a seemingly harmless piece of code can produce unexpected results or even throw errors. After all, the language was initially created in just 10 days. Why JavaScript on the Backend is Not That GreatJavaScript is a popular and versatile programming language that

WirekatMarkus

However, there is a third way of declaring variables in JavaScript, which is not using any keyword at all. This creates a variable that is global-scoped. This means that the variable is accessible everywhere in the code, even outside of the function where it is declared. For example:

function foo() {
  x = 1; // this is a global-scoped variable
}
foo();
console.log(x); // this will print 1, because x is accessible here

This may seem convenient, but it can also lead to some unexpected behavior and errors. One such example is the undefined variable puzzler, which involves a global variable that is declared without a keyword, but is used before it is assigned a value. Consider the following code:

function foo() {
  console.log(x); // what will this print?
  x = 1;
}
foo();

What do you think will happen when you run this code? Will it print 1, or will it throw an error?

The answer is… neither! Instead, it will print undefined! This is because of a feature of JavaScript called hoisting. Hoisting is the process of moving variable declarations to the top of their scope, before the code execution. This means that the code above is equivalent to:

function foo() {
  var x; // this is hoisted to the top of the function scope
  console.log(x); // this will print undefined, because x has not been assigned a value yet
  x = 1; // this will assign a value to x
}
foo();

However, hoisting only applies to the declarations, not the assignments. This means that if you declare a variable without a keyword, it will be hoisted to the top of the global scope, not the function scope. This means that the code above is actually equivalent to:

var x; // this is hoisted to the top of the global scope
function foo() {
  console.log(x); // this will print undefined, because x has not been assigned a value yet
  x = 1; // this will assign a value to x
}
foo();

This explains why the code prints undefined, instead of throwing an error. The variable x is declared in the global scope, but it is not assigned a value until the function foo is called. Therefore, when the function tries to print the value of x, it prints undefined, which is the default value of an uninitialized variable.

The solution to this puzzler is simple: always use a keyword to declare your variables, preferably let or const, which are block-scoped and prevent accidental reassignment or redeclaration. This way, you can avoid this puzzler and create variables that are clear and consistent.

JavaScript may be a language full of surprises, but with some attention to detail, you can avoid some of the most common pitfalls and write code that is robust and reliable.

Why JavaScript on the Backend is Not That Great

JavaScript is a popular and versatile programming language that powers many web applications. It is widely used for front-end development, where it can manipulate the Document Object Model (DOM), handle user interactions, and communicate with web servers. However, in recent years, JavaScript has also expanded its reach to the back-end,

WirekatMarkus

How to Use Regular Expressions in JavaScript

Regular expressions are patterns that can be used to match, search, or replace text in a string. Regular expressions can be very useful for validating user input, extracting data, or manipulating text. In JavaScript, regular expressions are objects that can be created using the RegExp constructor or the literal notation.

WirekatMarkus

JavaScript is a language full of surprises. Sometimes, a seemingly harmless piece of code can produce unexpected results or even throw errors. After all, the language was initially created in just 10 days.

Why JavaScript on the Backend is Not That Great

JavaScript is a popular and versatile programming language that powers many web applications. It is widely used for front-end development, where it can manipulate the Document Object Model (DOM), handle user interactions, and communicate with web servers. However, in recent years, JavaScript has also expanded its reach to the back-end,

WirekatMarkus

One such example is the missing comma puzzler, which involves an object literal with a missing comma between two properties.

Consider the following code:

var obj = {
  a: 1
  b: 2
};

What do you think will happen when you run this code? Will it create an object with two properties, a and b, or will it throw a syntax error?

The answer is… neither! Instead, it will create an object with a single property, a, whose value is a labelled statement.

A labelled statement is a way of naming a statement so that it can be referenced by a break or continue statement. For example:

loop1: // this is a label
for (let i = 0; i < 10; i++) {
  if (i === 5) {
    break loop1; // this will break out of the loop with the label loop1
  }
  console.log(i); // this will print 0, 1, 2, 3, and 4
}

In the case of this puzzler, the label is b and the statement is 2. This means that the object obj is equivalent to:

var obj = {
  a: b: 2
};

This is a valid syntax, but it is not very useful. The label b has no effect on the value of the property a, which is still 2. Moreover, the label b is not accessible outside of the object literal, so it cannot be used by a break or continue statement.

What's the best practice?

The solution to this puzzler is simple: always use commas to separate properties in an object literal. Alternatively, you can use the ES6 syntax of shorthand property names, which does not require commas. For example:

let a = 1;
let b = 2;
let obj = { a, b }; // this is equivalent to { a: a, b: b }

This way, you can avoid the missing comma puzzler and create objects that are clear and consistent. JavaScript may be a language full of surprises, but with some attention to detail, you can avoid some of the most common pitfalls and write code that is robust and reliable.

Check out another similar JavaScript puzzler below!

The case of global-scoped variables in JavaScript

JavaScript is a language that allows you to declare variables in different ways. You can use the var keyword, which creates a variable that is function-scoped. This means that the variable is accessible within the function where it is declared, and any nested functions. For example: function foo() { var x

WirekatMarkus

JavaScript is a popular and versatile programming language that powers many web applications. It is widely used for front-end development, where it can manipulate the Document Object Model (DOM), handle user interactions, and communicate with web servers. However, in recent years, JavaScript has also expanded its reach to the back-end, where it can run on platforms like Node.js and Express.js, and interact with databases, APIs, and other services. This trend of using JavaScript for both front-end and back-end development is known as full-stack JavaScript.

While full-stack JavaScript may seem appealing for its simplicity, consistency, and convenience, it also comes with many drawbacks and challenges. In this article, I will argue that JavaScript on the backend is not that great, and that developers should consider other alternatives for their back-end needs. I will base my argument on three main points: performance, security, and maintainability.

Why do companies choose JavaScript in their servers anyway?

There are loads of examples of some popular companies switching or start off with let's say, Node.js - but ultimately having to use other solutions, further fragmenting their codebase stack. Here are some examples.

Netflix

A company that wants to build a high-performance and scalable web application that can handle millions of concurrent requests and stream large amounts of data, such as a video streaming service or a social media platform. They may choose to use JavaScript on their backend because of its popularity, simplicity, and convenience, as well as its ability to run on both the front-end and the back-end.

However, they may face challenges with JavaScript’s poor performance, single-threaded nature, and lack of standardization and optimization. They may be better served by another language that offers faster execution, multi-threading, and more mature and optimized libraries and frameworks, such as C#, Java, Kotlin or Ruby.

For example, Netflix switched from Java to Node.js on their backend in 2015, but they still faced performance and reliability issues, and they had to use other technologies, such as Java, Python, and Go, to complement their Node.js backend.

PayPal

A company that wants to build a secure and reliable web application that can handle sensitive and confidential data and transactions, such as a banking or e-commerce service. They may choose to use JavaScript on their backend because of its flexibility, expressiveness, and dynamism, as well as its ability to create interactive and responsive user interfaces. However, they may face risks with JavaScript’s lack of security and reliability, dynamic typing, and loose error handling and logging. They may be better served by another language that offers more security and reliability, static typing, and robust error handling and logging, such as C#, Java, Python, or Ruby. 

For example, PayPal switched from Java to Node.js on their backend in 2013, but they still had to deal with security and reliability issues, and they had to use other technologies, such as Java, C++, and Scala, to supplement their Node.js backend.

LinkedIn

A company that wants to build a maintainable and readable web application that can handle complex and evolving business logic and requirements, such as a CRM or ERP service. They may choose to use JavaScript on their backend because of its flexibility, expressiveness, and dynamism, as well as its ability to support different styles and paradigms of programming. However, they may face difficulties with JavaScript’s low maintainability and readability, inconsistency and incompatibility, and constant evolution and obsolescence. They may be better served by another language that offers more maintainability and readability, structure and standardization, and stability and maturity, such as C#, Java, Kotlin or Ruby.

For example, LinkedIn switched from Ruby on Rails to Node.js on their backend in 2011, but they still had to deal with maintainability and readability issues, and they had to use other technologies, such as Scala, Java, and Python, to augment their Node.js backend.

The drawbacks of JavaScript on the server

Performance

One of the main disadvantages of using JavaScript on the backend is its poor performance compared to other languages. JavaScript is an interpreted language, which means that it is executed by a JavaScript engine at runtime, rather than being compiled into machine code beforehand. This adds an extra layer of overhead and slows down the execution speed. Moreover, JavaScript is single-threaded, which means that it can only handle one task at a time, and relies on asynchronous callbacks and promises to deal with concurrent operations. This can lead to complex and messy code, as well as potential memory leaks and performance bottlenecks.

In contrast, other languages like C#, Java, Python, or Ruby are either compiled or have faster interpreters, and support multi-threading or multi-processing, which allow them to handle multiple tasks simultaneously and efficiently. These languages also have more mature and optimized libraries and frameworks for back-end development, such as ASP.NET, Spring Boot, Django, or Rails, which offer better performance and scalability than JavaScript-based solutions.

Security

Another major drawback of using JavaScript on the backend is its lack of security and reliability. JavaScript is a dynamically typed language, which means that it does not enforce strict type checking and allows variables to change their types at runtime. This can lead to unexpected errors and bugs, as well as make the code more vulnerable to malicious attacks, such as injection, cross-site scripting (XSS), or denial-of-service (DoS). Furthermore, JavaScript does not have a standard way of handling errors and exceptions, and relies on the developer to implement proper error handling and logging mechanisms. This can result in silent failures and unhandled errors, which can compromise the functionality and integrity of the back-end application.

On the other hand, other languages like C#, Java, Python, or Ruby are either statically typed or have optional type annotations, which help to catch errors and bugs at compile time or runtime, and prevent type-related vulnerabilities. These languages also have built-in or standardized ways of handling errors and exceptions, such as try-catch-finally blocks, raise-rescue blocks, or decorators, which make the code more robust and reliable. Additionally, these languages have more secure and trustworthy libraries and frameworks for back-end development, such as ASP.NET Core, Spring Security, Flask, or Sinatra, which offer features like authentication, authorization, encryption, hashing, or CSRF protection, which enhance the security and reliability of the back-end application.

Maintainability

A final disadvantage of using JavaScript on the backend is its low maintainability and readability. JavaScript is a flexible and expressive language, which means that it allows the developer to write code in different styles and paradigms, such as imperative, functional, or object-oriented. However, this also means that there is no clear or consistent way of writing JavaScript code, and that different developers may have different preferences and conventions. This can lead to inconsistent and incompatible code, as well as make the code harder to read, understand, and debug. Moreover, JavaScript is a constantly evolving language, which means that new features and standards are added frequently, such as ES6, ES7, or ES8. This can lead to compatibility and compatibility issues, as well as make the code outdated and obsolete.

In comparison, other languages like C#, Java, Python, or Ruby are more structured and standardized, which means that they have clear and consistent ways of writing code, and that most developers follow the same or similar conventions and best practices. This can lead to consistent and compatible code, as well as make the code easier to read, understand, and debug. Furthermore, these languages are more stable and mature, which means that new features and standards are added less frequently, and that the code is more likely to remain relevant and up-to-date.

JavaScript brings its own set of awful quirks

You might have seen some memes of things that could only be found in JavaScript. The language has its own set of design flaws - after all, it's a well known fact that it's only been written in 10 days.

Numbers

If you're dealing with currency or payments, stay away from JavaScript. One of the quirks of JavaScript is that it does not have a separate data type for integers and floats. All numbers in JavaScript are represented as 64-bit floating-point numbers, following the IEEE 754 standard. This means that JavaScript can handle large and small numbers, as well as fractions and decimals, but it also means that JavaScript can have precision and rounding errors, especially when dealing with arithmetic operations.

For example, JavaScript cannot accurately represent some fractions, such as 0.1 or 0.2, and instead approximates them with binary fractions, such as 0.1000000000000000055511151231257827021181583404541015625 or 0.200000000000000011102230246251565404236316680908203125. This can lead to unexpected results, such as 0.1 + 0.2 !== 0.3, or 0.1 * 10 !== 1. These errors can accumulate and cause significant discrepancies in calculations and comparisons.

Moreover, JavaScript can have rounding errors when converting numbers to strings, or parsing strings to numbers, especially when using the built-in methods, such as toString, toFixed, toPrecision, parseInt, or parseFloat. These methods can either truncate, round up, or round down the numbers, depending on the radix, precision, or notation, and cause inconsistencies and inaccuracies.

For example, (0.1 + 0.2).toFixed(1) === '0.3', but (0.1 + 0.2).toFixed(2) === '0.30', or parseInt('2/888') === 2, but parseInt(2/888) === 0.

Falsy Logic

Another quirk of JavaScript is that it has a concept of falsy values, which are values that are considered false when used in a boolean context, such as an if statement, a logical operator, or a ternary operator. JavaScript has six falsy values: false, 0, "", null, undefined, and NaN. This means that any expression or variable that evaluates to one of these values will be treated as false, and any expression or variable that evaluates to anything else will be treated as true.

This can lead to confusing and misleading logic, especially when using the equality operator, the logical operators, or the ternary operator, which can coerce the operands to boolean values, and cause unexpected outcomes. For example, 0 == false, but 1 == true, or "" == false, but "0" == true, or null == false, but undefined == false, or NaN == false, but NaN != true.

Furthermore, JavaScript has a concept of truthy values, which are values that are considered true when used in a boolean context. JavaScript has an infinite number of truthy values, which are basically any values that are not falsy. This means that any expression or variable that evaluates to one of these values will be treated as true, and any expression or variable that evaluates to a falsy value will be treated as false.

This can also lead to confusing and misleading logic, especially when using the equality operator, the logical operators, or the ternary operator, which can coerce the operands to boolean values, and cause unexpected outcomes.

For example, "hello" == true, but "hello" != false, or [] == true, but [] != false, or {} == true, but {} != false, or function() {} == true, but function() {} != false.

What about TypeScript?

Some developers may argue that TypeScript, a superset of JavaScript that adds static type checking and other features, can solve some of the problems of JavaScript on the backend, such as type-related errors, bugs, and vulnerabilities.

However, TypeScript is only compile-safe, not runtime-safe, which means that it can only catch type errors and bugs at compile time, not at runtime. TypeScript still compiles to plain JavaScript, which means that it still inherits the dynamic and weak typing of JavaScript, and that it still relies on the JavaScript engine to execute the code. Therefore, TypeScript cannot guarantee the security and reliability of the code at runtime, and it cannot prevent runtime errors and bugs, such as type coercion, null pointer exceptions, or undefined values.

Moreover, TypeScript adds another layer of complexity and overhead to the development process, as it requires the developer to write and maintain type annotations, interfaces, and declarations, as well as to use transpilers, such as Babel or TypeScript itself, to convert the code into compatible JavaScript.

Although I truly love TypeScript when dealing with frontend code, it also has its own limitations and quirks, such as the lack of support for some JavaScript features, such as decorators, or the need to use type assertions, such as as or !, to override the type checker. TypeScript also has its own compatibility and interoperability issues, such as the need to use type definitions, such as @types, to use external libraries and modules, or the potential conflicts and inconsistencies between different versions and configurations of TypeScript.

The biggest issue is actually in the ecosystem

Another challenge of using JavaScript on the backend is its dependency on NPM, the default package manager for Node.js and JavaScript. NPM allows the developer to install and manage external libraries and modules, such as Express.js, or MongoDB as well as to create and publish their own packages. However, NPM is also a disaster that is insecure and prone to dependency hell, which means that it can cause security and reliability issues, as well as make the code hard to maintain and update.

For example, NPM is insecure because it does not verify or audit the packages that it hosts, and it allows anyone to publish or update any package, without any quality or security checks. This can lead to malicious or compromised packages, such as the infamous event-stream incident in 2018, where a popular package was hijacked and injected with malicious code that stole cryptocurrency wallets . NPM is also prone to dependency hell because it does not have a clear or consistent way of resolving and managing dependencies, and it allows packages to have multiple and nested dependencies, which can create conflicts and inconsistencies. This can lead to bloated and fragile code, such as the notorious node_modules folder, which can contain thousands of files and folders, and take up gigabytes of space .

In contrast, other languages like C#, Java, or Ruby have more secure and reliable package managers, such as NuGet, Maven/Gradle, or RubyGems, which verify and audit the packages that they host, and have quality and security standards and policies. These languages also have more manageable and consistent ways of resolving and managing dependencies, and have flat and explicit dependency trees, which avoid conflicts and inconsistencies. These languages also have more compact and robust code, which do not rely on external packages for basic functionality, and do not create massive and unwieldy folders and files.

Even the person who brought JavaScript outside of the web regrets some decisions around it

NodeJS is not perfect. In fact, even the creator of NodeJS, Ryan Dahl, has admitted that he regrets some of the decisions he made when designing and developing NodeJS. Some of these regrets include:

• Not sticking with promises: Promises are a way of handling asynchronous operations in JavaScript, which allow the developer to write cleaner and simpler code, and avoid the callback hell, which is a common problem in NodeJS. Promises also enable the use of async/await, which is a syntactic sugar that makes the code look more synchronous and readable. Dahl added promises to NodeJS in 2009, but removed them in 2010, because he wanted to keep NodeJS minimal and simple. However, he later realized that promises were a better way of dealing with asynchronous code, and that they could have improved the usability and stability of NodeJS.
• Not using a standard and secure package manager: NPM is the default package manager for NodeJS, which allows the developer to install and manage external libraries and modules. However, NPM is also insecure and unreliable, because it does not verify or audit the packages that it hosts, and it allows anyone to publish or update any package, without any quality or security checks. This can lead to malicious or compromised packages, such as the event-stream incident in 2018, where a popular package was hijacked and injected with malicious code that stole cryptocurrency wallets. NPM is also prone to dependency hell, because it does not have a clear or consistent way of resolving and managing dependencies, and it allows packages to have multiple and nested dependencies, which can create conflicts and inconsistencies.
• Not using a consistent and explicit module system: NodeJS uses a module system that allows the developer to organize and reuse the code, and to import and export modules using the require and module.exports functions. However, this module system is inconsistent and implicit, because it does not follow the standard and widely used ES6 modules, which use the import and export keywords, and it does not require the developer to specify the file extension or the index.js file when importing a module. This can lead to confusion and ambiguity, as well as compatibility and interoperability issues, especially when working with browser-based JavaScript, which uses ES6 modules.

These are some of the regrets that Dahl has expressed about NodeJS, and some of the reasons why he has created a new project called Deno, which is a secure and modern runtime for JavaScript and TypeScript, that aims to address some of the flaws and drawbacks of NodeJS. Deno is still in development, and it is not compatible with NodeJS, but it offers some interesting and promising features, such as:

• Security by default: Deno does not allow the script to access any system resources, such as the network, the file system, or the environment variables, unless the developer explicitly grants the permission using flags, such as --allow-net or --allow-write. This prevents the script from performing any malicious or harmful actions, and protects the system from potential attacks.
• TypeScript support: Deno supports TypeScript out of the box, without requiring any configuration or transpilation. TypeScript is a superset of JavaScript that adds static type checking and other features, which can help to catch errors and bugs at compile time, and improve the readability and maintainability of the code.
• Simpler and standard module system: Deno uses a simpler and standard module system, that follows the ES6 modules, and that requires the developer to use absolute or relative URLs, with file extensions, to import and export modules. Deno also caches the remote modules locally, and does not rely on external package managers, such as NPM, to install and manage dependencies.

Conclusion

In conclusion, I have argued that JavaScript on the backend is not that great, and that developers should consider other alternatives for their back-end needs. I have shown that JavaScript on the backend suffers from poor performance, lack of security and reliability, and low maintainability and readability, compared to other languages.

One might say that a language is just a tool. It is. It's a tool that helps make web pages interactive, as JavaScript was made explicitly for the web. It's not a shortcut for backend development, or writing horrible, heavy Electron apps.

Therefore, I suggest that developers should use JavaScript for front-end development, where it excels, and use other languages for back-end development, where they offer more advantages and benefits. This way, developers can leverage the strengths of each language, and create more performant, secure, reliable, maintainable, and readable web applications.