Stale Element Reference Exception only while using Actionchains

I have been trying to select an option from a tree using below code. the element becomes visible after clicking on category and expanding the dropdown

    category= driver.find_element_by_xpath("xpath")
    category.click()
    s_element=driver.find_element_by_xpath("xpath")
    driver.execute_script("arguments[0].scrollIntoView();", s_element)
    action.move_to_element(s_element).perform()
    action.double_click(s_element).perform()


I get the following error: "selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: element is not attached to the page document"

This only happens when I use actionchains to context click/ double click. suppose in the same code above i try to just click on the element it works just fine(and scrollintoview works fine as well).

    category= driver.find_element_by_xpath("xpath")
    category.click()
    s_element=driver.find_element_by_xpath("xpath")
    driver.execute_script("arguments[0].scrollIntoView();", s_element)
    s_element.click()

Not able to make out what the issue is. The only two ways i can add the s_element item is either by double-click or right click-->add. What is the reason? solution/workaround?? Thanks!