Solving the Task got bad yield Error in Python Asyncio with Treq

Learn why you encounter the `Task got bad yield` error in Python when using asyncio with Treq, and discover effective solutions using aiohttp.
---
This video is based on the question https://stackoverflow.com/q/68732435/ asked by the user 'vector8188' ( https://stackoverflow.com/u/1234419/ ) and on the answer https://stackoverflow.com/a/68740302/ provided by the user 'Artiom Kozyrev' ( https://stackoverflow.com/u/10311377/ ) at 'Stack Overflow' website. Thanks to these great users and Stackexchange community for their contributions.

Visit these links for original content and any more details, such as alternate solutions, latest updates/developments on topic, comments, revision history etc. For example, the original title of the Question was: python: Task got bad yield:

Also, Content (except music) licensed under CC BY-SA https://meta.stackexchange.com/help/l...
The original Question post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license, and the original Answer post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license.

If anything seems off to you, please feel free to write me at vlogize [AT] gmail [DOT] com.
---
Understanding the Task got bad yield Error in Python

If you're working with Python 3.6.7 and attempting to make parallel requests using asyncio and treq, you might have encountered the perplexing error: Task got bad yield. This issue arises when there is a mismatch between the asynchronous framework you are using and the libraries involved in making HTTP requests. This article aims to clarify the problem and provide a comprehensive solution.

The Problem Explained

The root of the Task got bad yield error lies in the incompatibility between asyncio and the treq library. treq is a library that relies on the Twisted framework, which has its own event loop and does not play nicely with asyncio. When you attempt to use these two together, asyncio cannot properly manage the Deferred objects created by treq, leading to the aforementioned error.

Error Log Breakdown

When you run your code, you might see an error log similar to this:

[[See Video to Reveal this Text or Code Snippet]]

This log indicates that something went wrong during the await operation, particularly when handling the treq.get calls.

The Solution: Switch to aiohttp

Given the situation, the most effective solution is to transition from using treq to aiohttp. The aiohttp library is specifically designed to work with asyncio, making it a better fit for your use case of making parallel HTTP requests. Below, we will guide you through the steps to revise your code.

Step-by-Step Code Correction

Install Aiohttp: If you haven't already installed aiohttp, you can do so using pip:

[[See Video to Reveal this Text or Code Snippet]]

Update Your Code: Replace the treq library usage with aiohttp. Here is a refactored version of your original code:

[[See Video to Reveal this Text or Code Snippet]]

Key Changes Explained

Imported aiohttp: Instead of treq, we now use aiohttp for making HTTP requests.

Created an Async Client Session: We utilize aiohttp.ClientSession() to manage our connections efficiently.

Implemented Fetch Function: We created a separate fetch_service function to handle the request for each URL.

Utilized asyncio.run(): This is the modern way to run the main async function as of Python 3.7+ .

Conclusion

By switching to aiohttp, you can effectively resolve the Task got bad yield error and smoothly make parallel HTTP requests. This change not only mitigates the error but also enhances the overall efficiency of your asynchronous operations in Python. If you have any further questions or need assistance with specific examples, feel free to ask!