Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Load Mozilla cookies.sqlite #686

Merged
merged 5 commits into from
Oct 17, 2015
Merged

Load Mozilla cookies.sqlite #686

merged 5 commits into from
Oct 17, 2015

Conversation

soimort
Copy link
Owner

@soimort soimort commented Oct 5, 2015

The old-fashioned Netscape cookies.txt is nowhere to be seen in modern browsers. We should really move on to a modern format (e.g. Mozilla's SQLite-based cookies), so there will be no need for one to manually export cookies.txt anymore.

Also, this will allow for easier downloads of Bilibili's member-only videos, just grab a cookie file from your Firefox:

$ you-get -i -c ~/.mozilla/firefox/gggfz9xl.default/cookies.sqlite \
  http://www.bilibili.com/video/av2987444

Bilibili uses CAPTCHA verification as part of its login process, so there is no way to automatize this like we do in Niconico. Preloading existed cookies is a must in such occasions.

Review on Reviewable

@soimort
Copy link
Owner Author

soimort commented Oct 5, 2015

The plan is to support the old plaintext cookies.txt, Mozilla cookies.splite and Chrome Default/Cookies. I'm not merging this til we're fully done.

@soimort
Copy link
Owner Author

soimort commented Oct 17, 2015

Implementing Chromium cookies is non-trivial job - the encrypted_value stored in SQLite was encrypted using AES (therefore we need a 3rd party crypto library like pycrypto to handle the algorithmic thing).

Even more sophisticated, based on this article http://n8henrie.com/2013/11/use-chromes-cookies-for-easier-downloading-with-python-requests/, the encryption methods differ on Linux and OS X (didn't mention Windows).

Forget about Chrome, fair enough to support moz_cookies and cookies.txt only.
(To my knowledge, not only Firefox, but also some WebKit based browsers such as GNOME Web, makes use of moz_cookies too)

@soimort soimort merged commit 67a6b9b into develop Oct 17, 2015
@soimort soimort deleted the cookies-sqlite branch October 17, 2015 03:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant