After login, don't logout and scrape data with actions, skip blocks?

How to do it?

Imagine an site that you need login to navigate into dashboard pages.
But this site recognize excessive logins and logouts, so you will need login and stay.

It is just do not close browser, but how to

Working flow blocks:

1- Navigate to URL
2- Login and submit login form
2- Navigate into dashboard page
3- type on input from sliced
4- click
5- keycheck

until here all good, but how to execute steps 2-5 (skipping the first two steps, responsible for the url login)

Is this even possible?

OB2 currently automatically closes the browser so the only way is to save the cookies in a global variable (and then restore them inside the new browser that is opened), check out this guide.

1 Like

Really make sense so I can maintain the cookies until the login expires, but I have some doubts

This should be inserted in loli code right?

So I need copy token from local storage

Paste in globals.token

  • But how to refresh token?

  • How to set logic to get the token?

I was wondering how that can be done, maybe sheduling a login submit job to store new tokens?
And if is right, how to parse the Output variable from GET COOKIES BLOCK (puppeteer) to a global variable and route this?

  • And about user, how to include?

An input on config tab to store this piece of code would be nice, I should request it on github?

Thanks bro

PS: After some searches i’ve found this:

But it’s php, OB2 isn’t compatible, is it?

You can get stuff from the local storage using an execute js block in puppeteer. So when you have to refresh the token it’s as simple as this.

return localStorage.getItem("@System:token");

What do you mean exactly? You can already use custom inputs to specify values at the start of a config

Otherwise if you mean to just take some notes, just put them in the readme section.

You cannot run PHP with OB2, or at least not in an easy way. You would have to set up xampp/lampp locally and host your script, then call it from OB2 with an http request.

I will need 2 configs right? one to store token and other to scrap data
If I manually copy token to set the token to globals.token variable it will canno’t be refreshed, right?

This probably should be wrong:


 // Logic to get the token goes here

    expression = $"return localStorage.getItem(\"@System:token\");"
  => CAP @stoken

  expression = $"return localStorage.getItem(\"@System:user\");"
  => CAP @suser


  // You need to set the token to the 'globals.Token' variable
  globals.Token = "@stoken","@suser";
}

How to do correctly?
I’m trying get token by javascript as you said and store in globals.token

In other config,
Which block should I use to call the “globals.Token” output ?

No you need only 1 config. Global variables are per-job so you cannot share them across jobs. The only way to make it work with 2 separate jobs is to pass data using a file.

Regarding the code, I said use an ExecuteJS block, that doesn’t look like a block. Add it using the stacker interface and then copy paste it in position. Output directly to the global variable

=> VAR @globals.Token

should work. Otherwise do

globals.Token = stoken;

same for the other one

globals.User = suser;

Then you can use those same variables below in your actual config, e.g. using interpolation <globals.Token>

Still wrong? I’ve created a execute JS Block, copied and pasted

PS: already added the ; on end of globals.user line

Here is the lolicode

image

// Make sure only 1 bot at a time enters this section of code
// by using an asynchronous lock
ACQUIRELOCK globals
TRY
bool needNewToken = true;

// Check if 'globals' has a 'Token' property
if (((IDictionary<String, object>)globals).ContainsKey("Token"))
{
  // If it does, it means Token and Counter already exist and we can use them
  // Check if the counter is below 1000
  if (globals.Counter < 1000)
  {
    // Perfect, we don't need a new token
    needNewToken = false;
  }
}

// If we do need a new token
if (needNewToken)
{
  // We need to set the Token and Counter properties of 'globals'
  // We (re)set the Counter to 0
  globals.Counter = 0;

  // Logic to get the token goes here

  expression = @return localStorage.getItem("@System:token");
  => VAR @globals.Token

  expression = @return localStorage.getItem("@System:user");
  => VAR @globals.User

  // You need to set the token to the 'globals.Token' variable
  globals.Token = globals.Token;
  globals.User = globals.User;
  
}

// Here we know for sure the Counter exists because we passed all the logic above
// Increase the counter by 1 for this attempt
globals.Counter++;

// Rethrow any exception
CATCH
throw;

// Exit the lock that we entered at the start
FINALLY
RELEASELOCK globals
END

// Normal config goes here, you are guaranteed to have a
// globals.Token you can use at this point, which is refreshed every 1000 attempts
LOG globals.Token

It looks good but you have to open the browser and navigate to the site to get the token, otherwise you’re doing executejs on a browser that doesn’t exist! So before the executejs, open the browser, navigate to the site, then executejs to get the token and user, and finally close the browser again. All this needs to be in the “Logic to get token goes here” section of the code.

So OPEN BROWSER and NAVIGATE TO should be on to top of loliscript?

like this

image

still getting error =\

image

No, you insert them inside

// Logic to get the token goes here

// PUT THEM HERE (take their lolicode representation, cut it and paste it here)

BLOCK:ExecuteJS
  expression = "return localStorage.getItem('@System:token')";
  => VAR @globals.Token
ENDBLOCK

BLOCK:ExecuteJS
  expression = "return localStorage.getItem('@System:user')";
  => VAR @globals.User
ENDBLOCK

  globals.Token = globals.Token;
  globals.User = globals.User;

A lot of things are wrong with your current script man…