If you’ve kept up with our recent blogs, you’ll know about our ongoing efforts at Engine Room to build a robust AI-powered Test Case and Test Script Generator that can change the lives of QA engineers the world over. Today, we’re excited to talk about another product that we think may serve as an excellent “vessel” for our tool.
Many of you are more than familiar with Atlassian, the Australian software company that’s become ubiquitous in tech project management with its apps like Jira and Confluence. Through its Marketplace and its Forge development platform, Atlassian has always offered to app devs like us an outstanding customer base and a rich suite of tools. Most recently, though, they’ve put out their own answer to the LLM revolution — Rovo Agents, customizable AI bots that we believe our testing tool is destined to be one of. Atlassian’s Rovo comes in three forms: Search, Chat, and Agents. Let’s dive in and explore what Rovo means for you and for Engine Room’s Test Case and Test Script Generator.
The simplest way to use Rovo might be Rovo Search. This feature connects your Jira and Confluence sites to each other and to third-party products like Google Drive, SharePoint, Slack, Loom, Figma, GitHub, and Microsoft Teams — wiping out the hours you spend switching between different apps to look for old information. Search through issues, team documentation, mockups, edit history, and coworker profiles all in one place. Rovo Search respects user permissions as well, meaning differently-roled team members may see different results.
Next up is Rovo Chat. While quite close to other AI chatbots you’re used to, Atlassian’s solution is conveniently tied into your Jira or Confluence workspace. Chat is constantly keeping your work in mind as it generates content, answers your questions, and even executes actions for you. Unless otherwise directed, it considers the project/space you’re currently looking at, what type of work your team does, and what third-party products you have connected. You might ask it to summarize the issues or documents created within a specific date window, remind you of your high-priority tasks that are due soon, or change the assignee on an issue. If its answer doesn’t fully help you, you can follow-up and refine your prompts as much as you want. Your conversation history is maintained for you to give a certain title if needed and re-read anytime for up to 30 days. Non-intrusive options are also available for each Chat response that allow you to send feedback about the response to Atlassian.
The feature that surprised me the most about Rovo may have been its Chrome Extension. This brings the power of both Rovo Chat and Search to any public websites you visit. Chat can sum up complex webpages — or compare and connect them with information found in your Atlassian organization! Search can quickly find answers across your Atlassian sites and linked third-party services — straight from your browser, without even needing to open any apps or sites.
Now, we get to the fun stuff. Rovo Agents are, in my opinion, best described the way Atlassian says it: “configurable AI teammates”. Think of them as specialized versions of Rovo Chat. They’re a game-changer for quickly executing repetitive or complex jobs. Many come pre-built, while others are given out or sold on the Atlassian Marketplace. Some Agents that are pre-built and customized by Atlassian include release notes drafters, project priority/progress trackers, issue inspectors that reference your organization’s quality standards, planners that break down large goals into reasonable issues, and so many more. All of these work with you in the Chat interface, so you can direct them, explain additional context, and give them feedback until you get the results you need. They’ll even fit into your Automation Rules, eliminating even those few clicks to generate content or take action!
But sometimes, you need something more tailored. Do you work in a rather specific company department, making it difficult to generate relevant answers, drafts, or summaries with ChatGPT? Got a workflow that’s unique to your team, but involves actions that require too many resources? You can build your own custom Rovo Agents anytime to handle and streamline these processes! You’re in control of the goals, the input, and the expected output, and the Agents intelligently follow their instructions. Duplicate an existing Agent if you just want to make slight modifications, create your own from scratch with the natural-language, No-Code interface, or go the whole nine yards by creating your own Code Agent built through Forge, Atlassian’s serverless development platform. The best part is that anyone can create as many as needed and share them privately, publicly, for free, or at a price on the Atlassian Marketplace!
Custom Agents excel with low-risk tasks that call for text output and take humans lots of time or repetition. If you choose to create your own Agent, you’ll be able to specify its permitted actions — should it create Jira issues, add Confluence comments, summarize an incident, and/or update an issue status? Additionally, you can define which Jira projects or Confluence pages it should reference as knowledge. Most importantly, you’ll need to provide it with an Instructions body — sort of like an all-in-one backstory, job description, and mini-handbook. Crafting Instructions, like learning to prompt-engineer with any chatbot or LLM, can take some trial and error until the Agent consistently produces results and performs actions to your liking. The time saved can be monumental for your team, though — because, unlike prompts, Instructions need only be saved once in order to apply to every single response. Don’t worry if you’re not experienced with prompt engineering or custom/fine-tuned models; if you reach out to us at Engine Room, I’ll be happy to drop any pointers I can!
So, back to our AI Test Case (and Test Script) Generator. You can probably guess the direction in which we’re heading: we’re excitedly investigating the possibility of building our “TCG” as an Atlassian Rovo Code Agent. As a customer, you’d be able to download it straight into your Jira and Confluence workflows, saving your team countless hours and endless stress with just a few clicks. As a Rovo Agent, the TCG would immediately know all about your organization’s way of work, the kind of testing you need to do, and how best to integrate those observations into its generated tests.
Rovo has hardly been in GA for a month, so there are exactly zero third-party Agents currently offering automated Test Case Generation. We think the stars aligned for Engine Room in terms of Rovo’s launch timing! That said, we do still have some decisions to make. While it would be very exciting for our TCG to take the form of a Rovo Agent, we need to consider the flexibility that comes with simply offering it as an Atlassian Forge AI app (without Rovo). One approach would integrate much more naturally with each of your organizations and ways of work, while the other could provide us with more flexibility to give you all even cooler features in the future. Either way, it seems increasingly likely that our product will be offered as an Atlassian Marketplace app, and we’re thrilled about it. That means even simpler integration for all of you, a larger audience for us, and a boatload of support and resources for both sides!
Engine Room’s upcoming AI Test Case and Test Script Generator excites us more with each passing day, especially thanks to our latest initiative to combine it with Atlassian’s thriving platform in some way. Are you as thrilled as we are about saving hours and cash for your QA (and entire) team? Stay tuned for my next blog entry about this project — and, if you missed them, take a glance at my last two posts: the first big reveal, and a breakdown of our observations of various popular LLMs!