-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Re-implement unit tests with Stata? #47
Comments
Hi @etiennebacher , I am super sorry that I did not respond earlier - I somehow missed the email github usually sends out when an issue is opened! It's particularly strange because I actually check all my repos basically every day... I have dropped the tests against Stata, but have replaced them with extensive tests against In one devel-branch, I have added some notes on how
Overall, I think you are right that adding a simple test suite against Stata-boottest might help convince some users that it is safe to use fwildclusterboot. Though I am somewhat reluctant to pay 800 Euros for a non-academic Stata license to simply run a few tests 😃 So it would be super awesome if you could run some Stata code for me! If you are indeed up for doing so, I could sketch some Stata code and tests against All the best, Alex |
I have opened a new branch, in which I have included the 'old' tests of |
Hello Alex, thanks for your answer, no worries for the delay, I'm not at all in a rush 😄 Of course I don't expect you to pay a Stata license to run a tests suite. I'm still ok to run the tests for Stata (while I still have a license), I will take a look at the branch with the "old" tests to see how it goes. Concerning the workflow, since running these tests in a CI environment is not possible, I suppose the best way to operate is that you tag me before each CRAN release so that I can run the tests and see if everything works. Of course the problem with this approach is that you can't test directly if your code changes break the tests with Stata, but I don't see another option for now. |
Hi @etiennebacher, I thought about this a little more, and I think it would in fact be best if you or I (in case I find access to a Stata licence) would run the wild cluster bootstrap via fwildclusterboot and Stata and simply copy the parameters produced by Stata into the R test file, e.g.
I don't think I have a contributing.md file (I should really add it), but in case you would implement these Stata tests, I'd be super grateful and would be more than happy to add you as a package contributor :) |
Hi @s3alfisc, good idea, it's just gonna some time to clean the test file to comment out the Stata code and copy the results from Stata, but I'll do it eventually. FYI, I ran the test file (with all the Stata code run through RStata) and so far all the tests pass, so well done! |
As you mentioned in #24 and in your blog post, you dropped tests comparing your package's outputs to Stata's outputs, the reason being that you don't have a Stata license anymore and that it can't run on CI.
I think that testing against Stata's output is quite essential for many people (and I would also be reassured by the equivalence between your package and Stata's implementation). Therefore, I'm wondering whether you could create a CSV file that contains a list of outputs created on Stata, and then compare this list to the outputs of your package. This list would have to be updated every time Stata's boottest is updated but that shouldn't happen too frequently. For example,
fixest
does something like that for benchmarkingfeols
againstreghdfe
so maybe it's something that could be done for testing.I don't think I'm fluent enough with Stata to create such a list of outputs, but if you have some code and need someone to run it for you, just let me know
The text was updated successfully, but these errors were encountered: