Issue 140: Experiments using ChatGPT to enhance software testing
🕒 5 MIN READ | Published on: August 14, 2024This is the Software Testing Notes, a newsletter that goes out every Wednesday. I republish it here for sharing and referencing, but if you'd like to sign up you can do so right here:
Hello there! 👋
Welcome to the 140th edition of Software Testing Notes, a weekly newsletter featuring must-read content on Software Testing. I hope this week has been good for you so far.
This week, we will explore:
- How Uber Tests Payments In Production
- Experiments using ChatGPT to enhance software testing
- Test LLM outputs with promptfoo
- API Testing with Playwright in Java
- Penetration Testing on MYSQL
- and more...
🙏 Sponsored by
Qase — Combine all your testing activities into a single workspace
Manage manual and automated tests with 30+ seamless integrations, including a bilateral Jira integration. Customizable, shareable dashboards provide data for every test step, giving you a comprehensive view of your entire test suite. Plus, you can rely on 99.9% uptime and 24/5 live customer support across all time zones.
👉 Start a free trial — no CC required!
📚 Testing
How Uber Tests Payments In Production by Alvaro Duran
In general, we avoid testing in production and use dedicated staging/test/dev environment for our testing activities. Alvaro Duran argues that real-world conditions reveal issues that can't be detected in controlled settings. The article details Uber's approach to payment system rollouts as a case study, demonstrating how they prioritize resiliency by rolling out features cautiously and monitoring them closely in production.
Explaining Exploratory Testing by Maaret Pyhäjärvi
Exploratory Testing is a technique to test software. Right? Well, maybe there’s more to it. In this article, Maaret Pyhäjärvi challenge conventional views of exploratory testing, and advocates for more integrated and context-driven approach, and combining exploratory testing with modern tools and automation in agile environments.
Unlocking the Potential of ChatGPT with C.L.E.A.R Prompts by Viv Richards
Viv Richards did some experiments using ChatGPT to enhance software testing processes, and came up with C.L.E.A.R(Context, Lead Instructions, Examples, Assumptions, and Review) heuristic that is designed to improve the effectiveness of ChatGPT by structuring prompts in a way that yields more relevant and useful responses, particularly for generating test ideas and automation scripts.
The Five “I”s of Great Testing by Cassandra H. Leung
What distinguishes a great tester? Cassandra H. Leung introduces five essential qualities that all begin with the letter "I": imagination, investigation, informing, illuminating, and improving.
Applying Liminal Thinking to Software Testing by Brie Ransom
Liminal thinking is the art of finding, creating and using thresholds to create change. It is a kind of mindfulness that enables you to create positive change.
Brie Ransom explores how the principles of liminal thinking can be applied to software testing to improve testing quality and effectiveness.
Testing Basic: Fragile Tests by Gil Zilberfeld
"fragile tests"—tests that produce unreliable or inconsistent results.
This article by Gil Zilberfeld argues that fragile tests indicate deeper issues with code quality and highlights the importance of developer and tester collaboration in test planning to ensure robust, trustworthy code.
The Quality Paradox: Why Great Software Defies Definition by Alexis Monroy Leone
The nature of quality is complex and subjective, and it cannot be reduced to simple metrics or checklists. Alexis Monroy Leone encourages viewing quality as an ongoing conversation among stakeholders, leading to software that is not only technically sound but also genuinely valuable and impactful.
🔍 Software Testing
⚙️ Automation
Working with WebView context in Android mobile automation test by shivahari p
Working with WebView has always been a little tricky when it comes to android automation with Appium. shivahari p provides a comprehensive guide for automating the validation of URLs in a WebView context, addressing common obstacles like context switching and handling Chrome's welcome screens.
How to automate your Prompt Engineering Matrix With Promptfoo by Stuart Thomas
promptfoo — is a a tool for testing and evaluating Large Language Model (LLM) output quality.
In this article, Stuart Thomas shares how to setup, configure and test LLM outputs with promptfoo, and analyze the test results.
Cutting the High Cost of Testing Microservices by Nocnica Mellifera
Having separate continuous running environment for testing has it’s own set of challenges such as higher costs, maintenance overhead, and scalability. Nocnica Mellifera suggests exploring alternative strategies like dynamic provisioning and request isolation to optimize testing processes and reduce costs in modern software development.
Dive into API Testing with Playwright in Java by Nikola
You are really not that into Javascript but still want to try out Playwright? Nikola provides a step-by-step guide on how to setup Playwright-based API testing framework in Java, along with the framework's structure, key components, and usage instructions, aiming to simplify API testing processes.
🔍 Test Automation
💨 Performance
Enhancing Memory Leak Detection for Black Box QEs in iOS by Fran Roig
Fran Roig has developed a simple and user-friendly Memory Monitor for tracking memory usage in iOS applications in real-time. Fran walks us through the demo on how to use this utility to capture and view memory snapshots directly in the app's UI.
Load Test your APIs with K6 by Vaibhav Patil
This article by Vaibhav Patil provides a comprehensive guide on installing K6, writing a basic load-testing script, and utilizing Grafana Cloud to run the tests and visualize the results.
🔍 Performance Testing
🌞 Accessibility
Automated Accessibility Testing is Not a Silver Bullet by Monika Prasad
Balancing automation and manual efforts in accessibility testing is important. As Monika Prasad points out while automated tools are efficient for identifying basic issues, they cannot replace the nuanced understanding required for comprehensive accessibility audits.
🔍 Accessibility Testing
🔒 Security
Penetration Testing on MYSQL (Port 3306) by Vinayak Chauhan
Vinayak Chauhan walks through the process of installing MySQL, configuring access, and conducting penetration testing using tools like Nmap, Hydra, and Metasploit to expose potential security weaknesses.
🔍 Security Testing
🛠️ Resources & Tools
Selenium-Kubernetes — Deploying Selenium Grid on Kubernetes is a complex endeavour. Vivek Bhatt has created a GitHub repository with pre-configured YAML files to effortlessly deploy a Selenium Grid on your Kubernetes cluster.
Placemark — A web-based tool for creating, editing, and visualizing map data, in a variety of formats including GeoJSON, KML, Shapefiles, CSV, and more.
Hemmelig.app — An online app that allows you to securely share a one-time use link for sharing a password, confidential message, or other form of private data.
Loop — An elegant window management app for Mac users that lets you effortlessly choose your window direction using a radial menu triggered by a simple key press, customizable according to your preferences.
📝 List of Software Testers
It's hard to find good articles, podcasts on Software Testing. Even hard to find people who create them. Are you also looking for amazing software testers to follow or read their content ? check out this page dedicated to software testers.
Do you also create content around Software Testing ? Submit yours here and I will add it to the list.
🎁 Bonus Content
📌 OTHER INTERESTING STUFF
- Feeling Unmotivated? Use “Skinner’s Law” to Get Yourself Back on Track
- Using Examples of Successful Brands to Overcome Team-Building Issues
- Master Git Branching with the Interactive Game “Learn Git Branching”
⭐ LAST WEEK'S MOST READ
- SOLID Principles for Testers: The Dependency Inversion Principle by Kristin Jackvony
- The challenges of end-to-end testing and how to face them by Chun Fei ⭕️ Lung
- QA myth busting: QA slows work down by Vitaly Sharovatov
😂 And Finally,
📨 Send Me Your Articles, Tutorials, Tools!
Wrote something? Send links via Direct Message on Twitter @thetestingkit (details here). If you have any suggestions for improvement or corrections, feel free to reply to this email.
Thanks to everyone for subscribing and reading!
Happy Testing!
Pritesh(@priteshusdadiya)
Did You Enjoy This?
Then consider joining the 2,638 other people getting the Software Testing Notes newsletter. It's a collection of fascinating finds from my week, about wide range of topics surrounding software testing and whatever else catches my interest.