In the ever-evolving landscape of software development, mastering the art of efficient testing has become an indispensable skill for engineers. Among the myriad of testing frameworks and tools, Sorry Cypress stands tall as a reliable companion in the quest for flawless software. Whether you’re a seasoned developer or a curious novice, this document is your gateway to unlocking the full potential of Sorry Cypress.

Get ready to dive into its intricacies, discover how it seamlessly enhances your testing workflow, and understand why it has earned its place as a vital asset in our engineering endeavors. So, let curiosity be your guide as we delve into the world of Sorry Cypress, where test stability and reliability reign supreme, and possibilities abound.

The purpose of this document is to help out Engineers who might work with Sorry Cypress, and therefore, will need to understand what it is, how it works, how to set it up, and why our company uses it. 

What is Cypress, and why do companies use it?

Cypress is an open-source testing framework used by companies to automate web application testing. It offers a simple syntax, fast execution, and real-time debugging. Cypress helps improve application quality and testing efficiency.

What is Sorry Cypress, and why do we use it?

In order to see the Cypress test results we need a dashboard, and Sorry Cypress does that job. It is an Open-Source tool and is very cost-effective because we have self-hosted it on our Virtual Machines in the cloud. Before using Sorry Cypress the company used Cypress Cloud which came with a cost therefore, we deployed our own dashboard.

Where can I find these tests in the code?

You can find these tests in the cypress directory inside the root of every project. 

We are done with the basics WHY’s and WHATS, so let’s move on to understanding the architecture of Sorry Cypress and how it communicates with the Cypress Agent.

Sorry Cypress Architecture

Integral components of Sorry Cypress:

There are four components of Sorry Cypress that contribute to its functionalities.

• Director Service
• API Service
• Web Dashboard
• Storage Service ( We use MINIO )

Director Service

This service is responsible for:

• parallelization and coordination of test runs
• integration with Slack, GitHub, MS Teams, and emitting generic WebHooks
• saving tests results
• generating signed upload URL for saving failed tests screenshots

Note: The director has to be configured w.r.t a storage mechanism, we have configured it to use Minio Storage. So, to understand the director configuration with Minio, see Minio Configuration options.

API Service

API Service is a simple GraphQL wrapper that exposes a convenient way to query the data stored by Director.

The service is only required as an interface for the Web Dashboard, but can be used to query the database and describe the internal data models.
Also, see API Configuration options.

Web Dashboard

The dashboard allows end-users to interact with sorry-cypress using a browser and to

• track test runs progress
• browser test results, videos, and failures screenshots
• set projects configuration like WebHooks, Slack, MS Teams, and GitHub integration
• create and delete entries (projects, runs)

Also, see Dashboard Configuration options.

Storage Service

Cypress comes with the ability to take screenshots and videos, whether you are running via cypress open or cypress run, even in CI.

We need remote storage to store the generated screenshots and videos. The director service will send a signed upload URL that the Cypress agent will use to upload the generated artifacts.

A signed URL is a URL that provides limited permission and time to make a request. Signed URLs contain authentication information in their query string, allowing users without credentials to perform specific actions on a resource.

Sorry-cypress integrates with the major remote cloud storage solutions:

• AWS S3
• Minio integration (via Minio S3 Gateway) that is compatible with
• Google Cloud Storage
• IBM COS
• NAS
• HDFS
• Azure Blob Storage

Sorry-Cypress Full Set Up

Sorry Cypress is a docker-based tool that runs on docker-compose. It is production ready and can be configured as per needs.

Below is the docker-compose configuration for the setup.

version: '3.6'
services:
  director:
    image: agoldis/sorry-cypress-director:latest
    hostname: director
    environment:
      DASHBOARD_URL: 'https://cypress.levvy.net'
      EXECUTION_DRIVER: '../execution/mongo/driver'
      MONGODB_URI: 'mongodb+srv://superuser:[email protected]'
      MONGODB_DATABASE: 'sorry-cypress-levvy'
      SCREENSHOTS_DRIVER: '../screenshots/minio.driver'
      GITLAB_JOB_RETRIES: 'false'
      MINIO_ACCESS_KEY: 'venturenox'
      MINIO_SECRET_KEY: '123qwe123!'
      #MINIO_ENDPOINT: '3.101.34.86'
      #MINIO_URL: 'https://3.101.34.86'
      MINIO_ENDPOINT: 'minio.levvy.net'
      MINIO_URL: 'https://minio.levvy.net'
      MINIO_PORT: '9000'
      MINIO_USESSL: 'false'
      MINIO_BUCKET: 'sorry-cypress-levvy'
      #MINIO_READ_URL_PREFIX: 'https://3.101.34.86/sorry-cypress-levvy'
      MINIO_READ_URL_PREFIX: 'https://minio.levvy.net/sorry-cypress-levvy'
      PROBE_LOGGER: 'true'
    ports:
      - 1234:1234
  api:
    image: agoldis/sorry-cypress-api:latest
    hostname: api
    environment:
      MONGODB_URI: 'mongodb+srv://superuser:[email protected]'
      MONGODB_DATABASE: 'sorry-cypress-levvy'
      APOLLO_PLAYGROUND: 'false'
      PORT: 4000
    ports:
      - 4000:4000
  dashboard:
    image: agoldis/sorry-cypress-dashboard:latest
    hostname: dashboard
    environment:
      GRAPHQL_SCHEMA_URL: 'https://cygraphql.levvy.net/graphql'
      GRAPHQL_CLIENT_CREDENTIALS: ''
      PORT: 8080
      CI_URL: ''
    ports:
      - 8080:8080
    depends_on:
      - api
  storage:
    image: minio/minio
    hostname: storage
    environment:
      MINIO_ACCESS_KEY: 'venturenox'
      MINIO_SECRET_KEY: '123qwe123!'
      #MINIO_SERVER_URL: 'http://3.101.34.86'
      MINIO_SERVER_URL: 'https://minio.levvy.net'
      MINIO_BROWSER_REDIRECT_URL: 'https://minio-console2.levvy.net'
    volumes:
      - ./data/data-minio-cypress:/data
    command: server /data --console-address ":9001"
    ports:
      - 9001:9001
      - 9000:9000
#  createbuckets:
#    image: minio/mc
#    depends_on:
#      - storage
#    entrypoint: >
#      /bin/sh -c "
#      /usr/bin/mc config host add myminio https://minio.levvy.net:9000 venturenox 123qwe123!;
#      /usr/bin/mc rm -r --dangerous --force myminio/sorry-cypress-levvy;
#      /usr/bin/mc mb myminio/sorry-cypress-levvy;
#      /usr/bin/mc policy set download myminio/sorry-cypress-levvy;
#      /usr/bin/mc policy set public myminio/sorry-cypress-levvy;
#      exit 0;
#      "

You can further explore different setups from this official Github repository.

Our instance is up and running, but before we start using it in our pipeline, the instance must be configured with NGINX as a reverse proxy and assigned an SSL certificate using CERTBOT as it is our standard practice.

You can read this article on Digital Ocean on how to configure NGINX as a reverse proxy, and this article to learn how to use CERTBOT for assigning SSL certificates to the domains.

Below is the configuration of NGINX that we currently use.

server {
   server_name cydirector.levvy.net;
   location / {
      proxy_pass http://localhost:1234;
      include proxy_params;
   }
    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/cydirector.levvy.net/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/cydirector.levvy.net/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
    
   server_name cygraphql.levvy.net;
   location /graphql {
   proxy_pass http://localhost:4000;
   include proxy_params;
   }

    listen [::]:443 ssl; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/cygraphql.levvy.net/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/cygraphql.levvy.net/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
   server_name minio-console2.levvy.net;
   
   location / {
      proxy_pass http://localhost:9001;
      include proxy_params;
  }

    listen [::]:443 ssl; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/minio-console2.levvy.net/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/minio-console2.levvy.net/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
   server_name minio.levvy.net;
   location / {
      proxy_pass http://localhost:9000;
      include proxy_params;
   }

    listen [::]:443 ssl; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/minio.levvy.net/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/minio.levvy.net/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
    if ($host = cydirector.levvy.net) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

   listen 80;
   listen [::]:80;
   server_name cydirector.levvy.net;
    return 404; # managed by Certbot

}
server {
    if ($host = cygraphql.levvy.net) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

   listen 80;
   listen [::]:80;
    
   server_name cygraphql.levvy.net;
    return 404; # managed by Certbot

}
server {
    if ($host = minio.levvy.net) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

   listen 80;
   listen [::]:80;
   server_name minio.levvy.net;
    return 404; # managed by Certbot

}

server {
    if ($host = minio-console2.levvy.net) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

   listen 80;
   listen [::]:80;
   server_name minio-console2.levvy.net;
    return 404; # managed by Certbot

}

The one thing to note here is that we are not using NGINX as an ingress to the Sorry Cypress dashboard. This is because we are using Cloudflare Zero Trust to secure it. I will put a reference to its documentation in the coming section.

Another important thing to note is that we have exposed the director service on the address “cydirector.levvy.net”. This is the URL we use in our CI/CD pipeline to run Cypress tests with the Sorry Cypress dashboard.

Cloudflare Set up

Now that we are done with setting up our Sorry Cypress instance with a fully configured NGINX reverse proxy, we now need to set up Cloudflare Zero Trust. These are the two major steps that encapsulate almost everything we need to do in order to set up Zero Trust and secure our Cypress Dashboard. The steps are as follows:

• Add the Web Application to Cloudflare ( Use this documentation )
• Create a tunnel ( Use this documentation )
  

After you are done setting it up, you will see the following output in the Zero Trust application and tunnel section.

Setup Cypress Step In Pipeline

This step has to be added inside the testing-preview step if needed. Below is an example implemented in Levvy.

Environment Variables:

RECORD_KEY: This can be a random value. It has meaning when using Cypress Cloud.

BASE_URL: This is the URL of the web application that needs to be tested.CYPRESS_API_URL: This is the URL of the directory service configured using NGINX.

Issues you may face in the complete setup

• The first issue you might encounter is uploading failure of videos and screenshots of failed tests. That might be for three reasons:
  • You might have misconfigured the minio server.
  • You might have misconfigured the director service that talks to minio.
  • You did not set up permissions on the minio bucket.
• The second issue might be the communication between backend services. It might be due to the following reasons:
  • Misconfigured NGINX
  • Misconfigured Endpoint URLS in docker-compose
• The third issue would be a failure of the test in the pipeline to initiate.
  • CYPRESS_API_URL is not correct
  • BASE_URL is not correct

Conclusion

As we conclude our journey through the fascinating realm of Sorry Cypress, we hope you now grasp the power it bestows upon engineers and testers alike. From its robust architecture that ensures stability to its seamless integration into modern workflows, Sorry Cypress has proven to be a game-changer in the world of software testing. Armed with this newfound knowledge, you are now equipped to harness its capabilities to elevate your testing practices and propel your projects towards success.