Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cookies expiring when changing IPs? #349

Open
ne1baf opened this issue Nov 13, 2024 · 5 comments
Open

Cookies expiring when changing IPs? #349

ne1baf opened this issue Nov 13, 2024 · 5 comments

Comments

@ne1baf
Copy link

ne1baf commented Nov 13, 2024

DO NOT DELETE THIS! Please take the time to fill this out properly. I am not able to help you if I do not know what you are executing and what error messages you are getting. If you are having problems with a specific video make sure to include the video id.

I'm using AWS Lambda to fetch transcripts, I recall this working correctly (not expiring after an IP change) with cookies a month or so ago. But now cookies seem to expire on IP change?
Note that if I hit refresh youtube, get a new cookie file and update the cookie file on my lambda and re-run the code, it will work again.

To Reproduce

  • get cookies with Cookie-Editor extension
  • run lambda code to fetch transcript with the cookies file (this should work)
  • wait 10 to 20 mins so that the instance disconnects
  • try running the same code again to fetch transcripts (this shouldn't work anymore)

What code / cli command are you executing?

For example: I am running

YouTubeTranscriptApi.get_transcript("yTocZOLjzC8", languages=['fr', 'en'], cookies='cookie.txt')

Which Python version are you using?

Python 3.11 on AWS Lambda

Which version of youtube-transcript-api are you using?

youtube-transcript-api 0.6.2

Expected behavior

I expected to receive the transcript

Actual behaviour

For example: Instead I received the following error message:

Could not retrieve a transcript for the video https://www.youtube.com/watch?v=yTocZOLjzC8! This is most likely caused by:

Subtitles are disabled for this video

If you are sure that the described cause is not responsible for this error and that a transcript should be retrievable, please create an issue at https://github.com/jdepoix/youtube-transcript-api/issues. Please add which version of youtube_transcript_api you are using and provide the information needed to replicate the error. Also make sure that there are no open issues which already describe your problem!
2024-11-13 09:22:42,541 - DEBUG - Response content analysis: playerResponse: True, captions: True, timedtext: False
Result: {'statusCode': 404, 'body': 'Failed to fetch transcript'}
@asonawalla
Copy link

FWIW, I'm seeing the same behavior in google cloud run (GCP's lambda equivalent) this week. I don't use the cookies feature at all and it was working consistently up to last week.

@jdepoix
Copy link
Owner

jdepoix commented Nov 15, 2024

Hi, this is very unlikely to be related to your cookies expiring, but more to YouTube blocking your lambdas IP address (look at #303 to find out more about this). Do you observe the same behaviour if not using cookie at all?

@asonawalla in your case your most definitely experience the issue described in #303.

@ne1baf
Copy link
Author

ne1baf commented Nov 15, 2024

I don't think it's about the lambda IP changing as I mentioned in my original message.

hi @jdepoix, yes I had a look at #303 but in my case trying to fetch transcripts from my lambda doesn't work if I'm not using cookies. However if I add cookies, I'll be able to make successful requests but after 10/20 mins, that cookie is not useable anymore and requests are blocked again. If I get a fresh cookie (even if it's from the same YT account as before), I'll be able to once again make successful requests for 10/20 mins.

I also tried this with a proxy that blocked by YT. Requests did not go through without using the cookie, however, after adding the cookie, requests would go through for 10/20 mins again before being blocked again.

Using another proxy without cookie fixed this for me but I used to be able to make requests with cookies without them "expiring" after 20 minutes and this behavior seems to have changed.

@asonawalla
Copy link

@jdepoix yeah, I figured. I'm running it on a non-cloud server without any issues now. Thanks for your work on this!

@peperojel
Copy link

this is not clear enough to reproduce

get cookies with Cookie-Editor extension
run lambda code to fetch transcript with the cookies file (this should work)

you need to specify how you are providing Cookies to Lambda

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants