lundi 27 avril 2020

KNN: "no missing values are allow" -> I do not have missing values

I am in a group project for a class and one of the people in my group ran the normalization, as well as creating the test/train sets so that we all have the same sets to work from (we're all utilizing different algorithms). I am assigned with running the KNN algorithm.

We had multiple columns with NA's so those columns were omitted (<-NULL). When attempting to run the KNN I keep getting the error of

Error in knn(train = trainsetne, test = testsetne, cl = ne_train_target,  :
  no missing values are allowed

I ran which(is.na(dataset$col)) and found:

which(is.na(testsetne$median_days_on_market))
# [1] 8038 8097 8098 8100 8293 8304

When I look through the dataset those cells do not have missing data.

I am wondering if I may get some help with how to either find and fix the "No missing values" or to find a work around (if any).

I am sorry if I am missing something simple. Any help is appreciated.

I have listed the code that we have below:

ne$pending_ratio_yy <- ne$total_listing_count_yy <- ne$average_listing_price_yy <- ne$median_square_feet_yy <- ne$median_listing_price_per_square_feet_yy <- ne$pending_listing_count_yy <- ne$price_reduced_count_yy <- ne$median_days_on_market_yy <- ne$new_listing_count_yy <- ne$price_increased_count_yy <- ne$active_listing_count_yy <- ne$median_listing_price_yy <- ne$flag <- NULL

ne$pending_ratio_mm <- ne$total_listing_count_mm <- ne$average_listing_price_mm <- ne$median_square_feet_mm <- ne$median_listing_price_per_square_feet_mm <- ne$pending_listing_count_mm  <- ne$price_reduced_count_mm <- ne$price_increased_count_mm <- ne$new_listing_count_mm <- ne$median_days_on_market_mm <- ne$active_listing_count_mm <- ne$median_listing_price_mm <- NULL

ne$factor_month_date <- as.factor(ne$month_date_yyyymm)
ne$factor_median_days_on_market <- as.factor(ne$median_days_on_market)

train20ne= sample(1:20893, 4179)

trainsetne=ne[train20ne,1:10]
testsetne=ne[-train20ne,1:10]

#This is where I start to come in

ne_train_target <- ne[train20ne, 3]
ne_test_target <- ne[-train20ne, 3]

predict_1 <- knn(train = trainsetne, test = testsetne, cl=ne_train_target, k=145)
# Error in knn(train = trainsetne, test = testsetne, cl = ne_train_target,  : 
#  no missing values are allowed

Mocha tests and require cache issues

I'm trying to somewhat separate my tests into unit, integration and qa_requires folders. So I though of this structure:

| /app
|  | ...
| /tests
|  | unit\
|  |  | file1.js
|  |  | file2.js
|  | integration\
|  |  | file1.js
|  |  | file2.js
|  | qa_requires\
|  |  | file1.js
|  |  | file2.js

AFAIK, there's a caching mechanism in node to handle requires as read here but, the files are in different folders and I can't tell what I'm doing wrong for node to avoid requiring all tests.

How it is possible to avoid a push in Github when the workflow tests fails?

I create the workflow to test my Python application before commit. The problem is that if the tests fail, the commit is pushed anyways. How I could add a condition to avoid the push if the test don't succeed?

Structure of the workflow file .yml here below.

`name: Python application on: push: branches: [ master ] pull_request: branches: [ master ]

jobs: build:

runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
- name: Set up Python 3.8
  uses: actions/setup-python@v1
  with:
    python-version: 3.8
- name: Install dependencies
  run: |
    python -m pip install --upgrade pip
    pip install -r requirements.txt
- name: Lint with flake8
  run: |
    pip install flake8
    # stop the build if there are Python syntax errors or undefined names
    flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
    # exit-zero treats all errors as warnings. The GitHub editor is 127 chars wide
    flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
- name: Test with pytest
  run: |
    pip install pytest
    pytest`

Test fail screenshot

Spring Boot: H2 Not Actually Retrieving From Database

I'm trying to write a simple Repository test in Spring Boot. The Test code looks like this:

public class UserRepositoryTest {
    private final TestEntityManager entityManager;
    private final UserRepository userRepository;

    @Autowired
    public UserRepositoryTest(TestEntityManager entityManager, UserRepository userRepository) {
        this.entityManager = entityManager;
        this.userRepository = userRepository;
    }

    @Test
    public void test() {
        String firstName = "Frank";
        String lastName = "Sample";
        String email = "frank@example.com";
        String username = "frank@example.com";
        String password = "floople";
        String passwordConfirm = "floople";

        RegisterUserRequest registerUserRequest = new RegisterUserRequest(firstName, lastName, email, username, password, passwordConfirm);
        User user = new User(registerUserRequest);
        user.setSpinsRemaining(0);
        userRepository.save(user);
        userRepository.setSpinsRemainingToTen();
        User found = userRepository.findByUsername(username);
        assertThat(found.getSpinsRemaining()).isEqualTo(10);
    }

What's I expect to happen is that the new User object is persisted to the database, the row in the database is modified to set spinsRemaining to 10, and then the now-modified row is retrieved from H2 and shoved into a new variable named "found". The "found" variable will point to an instance of a User object with ten spins remaining.

What actually happens is that the "found" variable points to the exact same instance of User that the "user" variable is. In fact, if I modify some property of the "user" variable AFTER persisting it to H2, the resultant "found" object also has the modified property. According to IntelliJ, both "user" and "found" are pointing to the same thing. How is that possible?

How to get reports from testcafe docker image?

I created an docker image based on the testcafe image.

But the xunit and html reports are not being generated, is there some way of doing it?

FROM testcafe/testcafe USER root RUN npm install -g testcafe-reporter-xunit RUN npm install -g testcafe-reporter-html WORKDIR /app COPY ./package.json /app COPY ./ /app CMD ["chromium:headless --no-sandbox", "--reporter xunit:reports/jReport.xml,html:reports/report.html"]

If I do a top inside the container looks like the command is correct:

/bin/sh /opt/testcafe/docker/testcafe-docker.sh chromium:headless --no-sandbox --reporter xunit:reports/jReport.xml,html:reports/report.html

Testing NativeScript application on emulator and device in parallel

I am working on angular NS app. I want to test 2 users behaviour by launching independent app on emulator and independent one on device. Currently i cannot find way (except duplicating project phisically) how to do this. I was trying to launch app on emulator and then via sidekick launch on device. It launches but device gets emualtors build and emulator stops syncing.

So is there a way how to do this?

Retrofit 2: Extract Value from Response Array

Im trying to extract a value from the response body of my Get Call in retrofit 2. I want to extract evey "id" yet I cant call the getters from the Response class, i think it could have something to do with the [] in the response class, when i remove the [] its returns a jackson error so i seem to be stuck

Call:

public interface RetrofitCallsGetFavourites {


@GET("diagnosis/configuration")
Call<GetFavouritesReponse[]> getDiagnosisFavourites (@Query("favourite") Boolean fave, 
@Header("Authorization") String authHeader);

}

This is the method im using, it wont let me call any getters

public class LoginApiStepDefinition extends TestBaseFix {

Retrofit retrofit = super.buildRetrofit(super.buildOkHttpClient());
RetrofitCallsLogin call = retrofit.create(RetrofitCallsLogin.class);
RetrofitCallsGetFavourites favecall = retrofit.create(RetrofitCallsGetFavourites.class);





@Then("I get the list of favourites with {string} and {string}")
public void iGetTheListOfFavouritesWithAnd(String username, String password) throws Exception {
    String auth = "Bearer " + LoginApi(username,password);
    String list = favecall.getDiagnosisFavourites(true,auth).execute().body().toString();
    System.out.println(list);

}
}

Response body class

public class GetFavouritesReponse {

@JsonProperty("id")
private Integer id;
@JsonProperty("version")
private Integer version;
@JsonProperty("createdDate")
private Object createdDate;
@JsonProperty("modifiedDate")
private Object modifiedDate;
@JsonProperty("rie")
private Boolean rie;
@JsonProperty("diagnosisName")
private String diagnosisName;
@JsonIgnore
private Map<String, Object> additionalProperties = new HashMap<String, Object>();

@JsonProperty("id")
public Integer getId() {
    return id;
}

@JsonProperty("id")
public void setId(Integer id) {
    this.id = id;
}

@JsonProperty("version")
public Integer getVersion() {
    return version;
}

@JsonProperty("version")
public void setVersion(Integer version) {
    this.version = version;
}

@JsonProperty("createdDate")
public Object getCreatedDate() {
    return createdDate;
}

@JsonProperty("createdDate")
public void setCreatedDate(Object createdDate) {
    this.createdDate = createdDate;
}

@JsonProperty("modifiedDate")
public Object getModifiedDate() {
    return modifiedDate;
}

@JsonProperty("modifiedDate")
public void setModifiedDate(Object modifiedDate) {
    this.modifiedDate = modifiedDate;
}

@JsonProperty("rie")
public Boolean getRie() {
    return rie;
}

@JsonProperty("rie")
public void setRie(Boolean rie) {
    this.rie = rie;
}

@JsonProperty("diagnosisName")
public String getDiagnosisName() {
    return diagnosisName;
}

@JsonProperty("diagnosisName")
public void setDiagnosisName(String diagnosisName) {
    this.diagnosisName = diagnosisName;
}

@JsonAnyGetter
public Map<String, Object> getAdditionalProperties() {
    return this.additionalProperties;
}

@JsonAnySetter
public void setAdditionalProperty(String name, Object value) {
    this.additionalProperties.put(name, value);
}


}

Here is an example of what is returned, the call is working correctly i just cant parse the informaiton correctly, im trying to extract all the id's

[ {
"id" : 15,
"version" : 0,
"createdDate" : null,
"modifiedDate" : null,
"rie" : false,
"diagnosisName" : "Angina pectoris - Angina pectoris with documented spasm"
}, {
"id" : 14,
"version" : 0,
"createdDate" : null,
"modifiedDate" : null,
"rie" : false,
"diagnosisName" : "Angina pectoris - Unstable angina"
},{
"id" : 22,
"version" : 0,
"createdDate" : null,
"modifiedDate" : null,
"rie" : false,
"diagnosisName" : "Seropositive rheumatoid arthritis - Seropositive rheumatoid arthritis, unspecified                 
Pelvic region and thigh"
}, {
"id" : 27,
"version" : 0,
"createdDate" : null,
"modifiedDate" : null,
"rie" : false,
"diagnosisName" : "Test Diagnosis Cardiology"
}, {
"id" : 26,
"version" : 0,
"createdDate" : null,
"modifiedDate" : null,
"rie" : false,
"diagnosisName" : "Test Diagnosis with Service"
} ]