You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have a problem with comments when scraping posts.
Doing this comment_full results in an empty list
dict_={}
for post in fs.get_posts(page_name,cookies=cookies, pages=2, timeout=60):
dict_= {"content_type":"facebook_post","post_id":post["post_id"], "text":post["text"],"post_text":post["post_text"],
"shared_text":post["shared_text"],"time":post["time"],"timestamp":post["timestamp"],
"images_lowquality_description":post["images_lowquality_description"],"likes":post["likes"],
"comments":post["comments"],"shares":post["shares"],"post_url":post["post_url"],
"link":post["link"],"links":post["links"],
"user_id":post["user_id"],"username":post["username"],"user_url":post["user_url"],
"comments_full":post["comments_full"],"reactors":post["reactors"]}
while doing this results in a full list:
CODE = str(code)
# get the post (this gives a generator)
gen = fs.get_posts(
post_urls=[CODE],
options={"comments": MAX_COMMENTS, "progress": True}
)
# take 1st element of the generator which is the post we requested
post_com = next(gen)
# extract the comments part
comments = post_com["comments_full"]
comment_list = []
# process comments as you want...
for comment in comments:
comment["comment_time"] = comment["comment_time"].strftime("%Y-%m-%d %H:%M:%S")
# e.g. ...get the replies for them
for reply in comment["replies"]:
reply["comment_time"] = reply["comment_time"].strftime("%Y-%m-%d %H:%M:%S")
comment_list.append(comment)
I would like to avoid returning to the same page several times so that I don't get banned. Can anyone help me with this?
The text was updated successfully, but these errors were encountered:
I have a problem with comments when scraping posts.
Doing this comment_full results in an empty list
while doing this results in a full list:
I would like to avoid returning to the same page several times so that I don't get banned. Can anyone help me with this?
The text was updated successfully, but these errors were encountered: