Pgoapi purge #3106

invisiblek · 2016-08-02T04:40:17Z

Kill our local copy of pgoapi and instead pull in a fresh copy from tejado's repo. Fixups along the way to handle the api changes.

medghaim · 2016-08-02T04:41:52Z

I know my +1 doesn't count, but thorough testing reveals all works fine and dandy. +1

invisiblek · 2016-08-02T04:42:15Z

"medghaim commented 2 minutes from now"

wizard!

invisiblek · 2016-08-02T04:44:57Z

People are going to need to re-run requirements.txt as well as probably purge their db (unless we want to bother with writing migration scripts)

medghaim · 2016-08-02T04:45:18Z

Ew no. Delete pogom.db, re pip install -r requirements.txt

(is this from the future too?)

invisiblek · 2016-08-02T04:46:16Z

"medghaim commented 2 minutes from now"

time traveler

Chlodochar · 2016-08-02T04:48:04Z

Related #671

medghaim · 2016-08-02T04:48:47Z

Yeah, invisiblek built off your earlier PR. Your commits are still up there somewhere!

chuyskywalker · 2016-08-02T04:59:02Z

pgoapi INFO level logging used to be silenced, I think.

Modify runserver.py with

    logging.getLogger('pgoapi.pgoapi').setLevel(logging.WARNING)
    logging.getLogger('pgoapi.rpc_api').setLevel(logging.INFO)

Chlodochar · 2016-08-02T05:27:02Z

Nice. I'm not able to test this soon, but would be very happy when this is merged! Should @tejado do a release of this commit?

FrostTheFox · 2016-08-02T05:30:53Z

Definitely runs. Going to compare pokemon amounts here in a few.

chuyskywalker · 2016-08-02T05:31:55Z

LGTM so far so good. I'll let it churn overnight, see if anything weird pops

medghaim · 2016-08-02T05:33:57Z

Always a good sign when things run, @FrostTheFox :D

invisiblek · 2016-08-02T12:23:48Z

Weird my hexes are off slightly now (they were perfectly aligned before). I'm using raw lat/long for them too, so they should be lined up.

chuyskywalker · 2016-08-02T13:20:41Z

Sometime during the overnight run I lost one of my search_worker threads. I expect it got some uncaught exception and dumped out, but my scrollback buffer is to short to see what happened.

invisiblek · 2016-08-02T13:29:16Z

I have 20 workers going, been running strong all night.

chuyskywalker · 2016-08-03T00:58:48Z

Here we go:


2016-08-02 14:15:44,595 [ search_worker_0][      auth_ptc][    INFO] PTC Login successful
Exception in thread search_worker_0:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/local/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/src/app/pogom/search.py", line 203, in search_worker_thread
    check_login(args, account, api, position)
  File "/usr/src/app/pogom/search.py", line 239, in check_login
    while not api.login(account['auth_service'], account['username'], account['password'], position[0], position[1], position[2], False):
  File "/usr/src/app/src/pgoapi/pgoapi/pgoapi.py", line 129, in login
    response = self.get_player()
  File "/usr/src/app/src/pgoapi/pgoapi/pgoapi.py", line 84, in function
    return request.call()
  File "/usr/src/app/src/pgoapi/pgoapi/pgoapi.py", line 182, in call
    response = request.request(self._api_endpoint, self._req_method_list, self.get_position())
  File "/usr/src/app/src/pgoapi/pgoapi/rpc_api.py", line 116, in request
    raise NotLoggedInException()
NotLoggedInException

That's causing the search_worker thread to dump out. Needs a bit of exception catching.

chuyskywalker · 2016-08-03T02:38:41Z

Alright, ignore my issue, it's not really a pgoapi upgrade problem but a "I didn't handle thread exceptions well enough" problem. I'll task that in a separate PR

chuyskywalker · 2016-08-03T03:52:58Z

LGTM -- I don't know if we hold off for the beehive misalignement or not though

invisiblek · 2016-08-03T03:54:19Z

nobody else is seeing the misalignment at all? the locations i'm using literally have the exact lat or long and they're showing up misaligned =/

@medghaim any idea here? (see screenshot from above)

chuyskywalker · 2016-08-03T04:14:44Z

I've fixed the reliability issues over in this a pr against this branch (that's the diff link excluding whitespace since I had to indent a lot for the fix). Should clear up and prevent any weird outliers for getting search threads lost.

Crazy-Drew · 2016-08-03T12:27:47Z

Can anyone give me good clear instructions on how to run more than 1 worker? simultaneously? That would be great!

I obviously am doing something wrong and can't get it working on my end.

- Handle exceptions better - Prevent search_workers from even fully exiting - High-load backoff is now backoff based instead of randomly assigned absolute - Logins will retry `login-retries` times before throwing an exception to cause the search_worker to create a new PgoAPI instance and start over

invisiblek · 2016-08-03T13:37:35Z

This fixes the hexes being off: tejado/pgoapi#171

invisiblek · 2016-08-03T15:58:10Z

pgoapi updated with the hex fix, latest commit pulls in that change

+111111111111111111

DomGrieco · 2016-08-03T16:04:17Z

@Crazy-Drew really not the place to be posting stuff like that

* Use standard logging for this and for the check of the old pogom/pgoapi * Automatically remove pogom/pgoapi, regardless if its empty or not. Its not going to be useful anymore and will cause problems.

AHAAAAAAA · 2016-08-03T16:49:26Z

This looks ready. If anyone has any objections, speak now or forever handle issues from here on out.

Chlodochar · 2016-08-03T17:34:47Z

Yay!

aldarund · 2016-08-03T17:40:00Z

@AHAAAAAAA @invisiblek /~https://github.com/AHAAAAAAA/PokemonGo-Map/issues/3370

hack3rjack · 2016-08-03T17:42:37Z

Also getting error when installing requirements.txt on line 18 for pgoapi, see gist

https://gist.github.com/hack3rjack/4a95b33dd811721d8b91389eeb44590f

aldarund · 2016-08-03T17:44:21Z

@hack3rjack its just a warnings that depends on your python build, all fine with it

hack3rjack · 2016-08-03T17:44:48Z

ok then, thanks @aldarund

Yossi · 2016-08-03T17:56:42Z

runserver.py

+    log.info("Done!")
+
+# Ensure user has updated to the new api
+newpgoapiPath = os.path.join(os.path.dirname(__file__), "src/pgoapi")


This is not where pip installs pgoapi. In fact, there is no src/ directory.

Better to simply try importing pgoapi and catching whatever errors come up. We can also check pgoapi.__version__if that is deemed necessary.

go ahead and use your super brain to improve it then

Will he is right, this check dont work and simply stop project from launching

Works perfectly fine here https://gist.github.com/invisiblek/f0e5f8b76eec1b23ebf1be3c5283072c

And? This is really braindead check. You basically rely that pip Will clone imto cuurent due and not clean it. This folder not even user, its just leftoff of installation . And recent pip Will delete this folder.

*due - directory
*User - used

Works awesome over here, definitely seeing way more pokemon now! I am loving this already!

Fixed in latest HEAD on develop.

In the future be a bit more respectful of people here. Calling anyone's code braindead is pretty fucking rude.

I'm sorry. That was in the heat of the moment as the code was crashing right as i needed it.

I'll edit that comment and it wont happen again.

invisiblek and others added 11 commits August 1, 2016 14:53

Remove unused scannedLabel() function

6ee3ad6

Remove pogom/pgoapi if empty at startup

d32af96

use pgoapi from requirements.txt instead of copying its files

66912bf

gitignore src/

ee5fa93

Fixups for using mainline pgoapi

a1499da

Update pgoapi to latest HEAD

69a7060

Use get_pos_by_name from pgoapi

b1bbe01

Update search api for new pgoapi

6aa9b26

Removing cell ID's, this functionality is provided in pgoapi

b06a1df

Update search for the new pgoapi (again)

7034b89

making sure we change with step location, rather than starting position

54e7d17

invisiblek assigned invisiblek, AHAAAAAAA, medghaim and Chlodochar Aug 2, 2016

invisiblek added this to the v2.2.0 milestone Aug 2, 2016

invisiblek added the refactor label Aug 2, 2016

silenced logging

24845a0

reyqn referenced this pull request Aug 3, 2016

Allow proxy usage (#2814)

8d21421

This was referenced Aug 3, 2016

develop: use pgoapi from requirements.txt instead of copying its files #671

Closed

Refactored PGoApi to make it immutable, PGoApiRequest introduced (solve missing pokemon) #1910

Closed

This was referenced Aug 3, 2016

Implement lured pokemon webhook #3299

Closed

Lured pokemon webhook #3302

Closed

Update pgoapi

b4640d9

Show a big scary warning if user hasn't updated

56cf029

* Use standard logging for this and for the check of the old pogom/pgoapi * Automatically remove pogom/pgoapi, regardless if its empty or not. Its not going to be useful anymore and will cause problems.

invisiblek merged commit 59bb500 into develop Aug 3, 2016

Yossi reviewed Aug 3, 2016
View reviewed changes

reyqn mentioned this pull request Aug 3, 2016

Fix beehive alignment #3305

Closed

6 tasks

AHAAAAAAA deleted the pgoapi-purge branch August 3, 2016 20:31

Pgoapi purge #3106

Pgoapi purge #3106

Conversation

invisiblek commented Aug 2, 2016

medghaim commented Aug 2, 2016

invisiblek commented Aug 2, 2016

invisiblek commented Aug 2, 2016

medghaim commented Aug 2, 2016 • edited Loading

invisiblek commented Aug 2, 2016

Chlodochar commented Aug 2, 2016

medghaim commented Aug 2, 2016

chuyskywalker commented Aug 2, 2016

Chlodochar commented Aug 2, 2016

FrostTheFox commented Aug 2, 2016

chuyskywalker commented Aug 2, 2016

medghaim commented Aug 2, 2016

invisiblek commented Aug 2, 2016

chuyskywalker commented Aug 2, 2016

invisiblek commented Aug 2, 2016

chuyskywalker commented Aug 3, 2016

chuyskywalker commented Aug 3, 2016

chuyskywalker commented Aug 3, 2016

invisiblek commented Aug 3, 2016

chuyskywalker commented Aug 3, 2016

Crazy-Drew commented Aug 3, 2016 • edited Loading

invisiblek commented Aug 3, 2016

invisiblek commented Aug 3, 2016

DomGrieco commented Aug 3, 2016

AHAAAAAAA commented Aug 3, 2016

Chlodochar commented Aug 3, 2016

aldarund commented Aug 3, 2016 • edited Loading

hack3rjack commented Aug 3, 2016

aldarund commented Aug 3, 2016

hack3rjack commented Aug 3, 2016

Yossi Aug 3, 2016 • edited Loading

Choose a reason for hiding this comment

invisiblek Aug 3, 2016

Choose a reason for hiding this comment

aldarund Aug 3, 2016

Choose a reason for hiding this comment

invisiblek Aug 3, 2016

Choose a reason for hiding this comment

aldarund Aug 3, 2016

Choose a reason for hiding this comment

aldarund Aug 3, 2016

Choose a reason for hiding this comment

Crazy-Drew Aug 3, 2016

Choose a reason for hiding this comment

invisiblek Aug 3, 2016

Choose a reason for hiding this comment

Yossi Aug 3, 2016 • edited Loading

Choose a reason for hiding this comment

medghaim commented Aug 2, 2016 •

edited

Loading

Crazy-Drew commented Aug 3, 2016 •

edited

Loading

aldarund commented Aug 3, 2016 •

edited

Loading

Yossi Aug 3, 2016 •

edited

Loading

Yossi Aug 3, 2016 •

edited

Loading