Forget fragile XPath or DOM selectors. AI-powered AgentQL finds elements reliably, even as websites change. Just specify what data you are scraping from the web with natural language-like queries, and AgentQL will handle the rest.Hey Product Hunters! 👋
I am Shuhao, cofounder of the AgentQL team. Allow me to introduce AgentQL, an AI-powered semantic framework designed to enable AI agents to seamlessly interact with the web, using natural language.
That's a lot of big words. Let's break down what AgentQL can do:
🕹️ Give it the command to find "price", and it will retrieve all the price data on an e-commerce page you spend less time building rigid selectors
🖥️ Take the same code to another site, and it can work just as well >> you spend less time maintaining fragile scripts
📝. Need to interact with sites as part of your work flow? Simply use natural language, tell AgentQL to click, scroll_page or fill.
So why did we decide to build AgentQL?
Traditional web automation struggles with identifying elements consistently through UI changes. When LLMs and agents first emerged, we envisioned a web infrastructure tailored for agentic interaction, which excels in handling these small but breaking changes, so we decided to tackle this challenge head-on.
How did we go about this?
At the heart of our framework is the AgentQL Query, a language crafted to describe locate web elements within a structured schema. We’ve integrated robust DOM processing with advanced prompt engineering, creating a powerful combination that dynamically generates context-aware prompts. By doing so, we overcome the fragility of static XPath or attribute selector-based scripts, enabling a more resilient and flexible web interaction infrastructure.
Would you like to join our journey to an agentic future?
Our early users have reported real productivity boosts with AgentQL, unlocking use cases that were previously too costly or complex to develop. Your feedback and questions will be invaluable to us.
👀 Please check out our technology at AgentQL.com 💬 Connect with our community on Discord and X.com
Thank you for being a part of our story!@shuhao_zhang Congratulations to the AgentQL team on their release! Great job - up and running in seconds🦾@shuhao_zhang Congrats on your launch day! Wishing you great success and new opportunities. What challenges did you overcome to get here?@shuhao_zhang @max_savonin1 Hi, Max! I may be able to help with some of these questions-- they're really great questions, btw!
- How does AgentQL ensure the accuracy of its natural language parsing and element identification? How easy is it to maintain AgentQL queries when websites undergo significant changes?
We do regular benchmarking to ensure our results are the best for accuracy and reproducibility, trading off against other dimensions. We also iterate against a set of real-world use cases to further improve AgentQL. We've found that providing context for certain terms is critical for accuracy and stabilization of results. As for maintaining AgentQL queries as websites undergo changes, we've found that we're generally able to handle most cases (even testing against some historic / archived versions of websites where available) so long as the elements exist, but obviously, if the structure changes too significantly, e.g. the elements or data no longer exists or the steps of a workflow are completely shuffled, the query wouldn't be able to adapt to this.
- Can AgentQL integrate with existing web scraping or automation frameworks? Can it learn and improve its understanding of user intent over time?
We currently provide an extension to the Playwright Python SDK, and are able to extend Page objects directly to run queries, interact, or provide some other utility functions. Most other frameworks that are leveraging Playwright where you still have access to the underlying Page object should be compatible. As for learning and understanding user intent, AgentQL itself is not personalized and doesn't adapt to usage, but we've provided tools like the aforementioned context to help disambiguate results.
- What security measures are in place to prevent unintended automation behavior or data breaches?
We currently view ourselves as a tool to enable developers, as it's simply a product which helps interpret data and provide elements for programmable interaction on web. We're curious to see the ingenuity of the community and believe our technology can be used to unlock a lot of good use-cases, but we will definitely remain vigilant and remain open to changing our position on this as we grow!
Hope that helps, and definitely open to connecting to explore possible collaborations!@antonikozelski Thanks so much! We're excited to hear that the onboarding is smooth for you. Appreciate the support!@kjosephabraham Thanks a lot! We’re excited about the launch and appreciate your well wishes. One of the biggest challenges we faced was ensuring the accuracy and reliability of our natural language queries across a wide range of websites. It took a lot of iteration and fine-tuning to make sure AgentQL could handle the diversity of web structures out there while still being user-friendly. Overcoming those hurdles has been incredibly rewarding, and we're thrilled to finally share it with the world!only log in option is google ? I will wait then. I have email // apple // github// google // but cannot use something that only is for google users ! Ping me when we can use other idsHey, Francoo! We'll have github SSO out by EOD-- stay tuned!@francoolaami just rolled out GitHub sign-in option! Thanks for the feedback!@francoolaami Thank you for the feedback and we enjoy building AgentQL together with the community. Looking forward to hearing more from you!AgentQL is probably the easiest web data extraction tool that I have ever used. While it doesn't require all the back and forth editing to choose the right web elements, it still offers the ability to use code to process the data extracted !
Give it a try and you won't regret!Thanks for giving us a shot and the kind words! We're really excited to see positive engagement from the community!Thank you for your support, Shawn!@shunxinpang Thanks for your shoutout and love to explore more towards the future of Agent with you!Congrats! It appears to be very similar to Semantic Targets invented by Open Agent Studio(open source) last year https://www.openagent.studio/ would you be able to improve semantic targets or is it essentially the same?Hi, Rohan! The high level concept with Semantic Targets sounds similar, but I believe that our underlying approach to resolving query elements is quite different. Though I myself have not tested Semantic Targets extensively, we've seen really good results on accuracy and reliable consistency across a broad range of real world use-cases using AgentQL. We also choose to return the control flow back to the developer rather than trying to have an LLM reason about and generate the full end to end task, so that developers can be confident in actions taken, any validations, and graceful error handling across the workflow.@murkypan Thanks for the explanation yes open agent studio also has an Agent API that allows explicitly triggering events directly using json actions in English. I'd encourage you to try it out and see if you can improve it.@rohan_arun1 Thanks! Will look it up!
