Skip to content

Implement invoice back-end

Uman Shahzad requested to merge uman/invoice into master

Setup

  1. Copy .environ/.env.dev to .env (if you already have an .env, do a merge with the new variables).
  2. Need JIRA credentials for a service account. Check our keypass DB for them, and assign them to JIRA_SERVICE_USER_USERNAME and JIRA_SERVICE_USER_PASSWORD properly. Make sure JIRA_SERVER_URL points to the proper JIRA domain (i.e. https://tasks.opencraft.com).
  3. Need a wkhtmltopdf binary, to do HTML-to-PDF conversion. See https://wkhtmltopdf.org/downloads.html.
  4. Need a PKCS12 file for Google Auth. This can be downloaded from the credentials page at Google Console, or from our secret accounting keypass DB (see our secrets repo; in there is the private key, and you can turn it into a PKCS12 file using the instructions). Name it .p12 (by default) and put it at the root of this repo. Or, change the GOOGLE_AUTH_PKCS12_FILE_PATH variable name in your environment accordingly if you've other plans. (FYI -- the password for the .p12 files should be notasecret by default; yes, it's really not a secret, as this is what Google automatically puts on all new PKCS12 files you download, and also what the PyDrive library expects to have to enter.)
  5. Set GOOGLE_AUTH_CLIENT_SERVICE_EMAIL to the service account email present in the accounting keypass DB.
  6. Need to set the proper GOOGLE_DRIVE_ROOT for file uploads -- see the accounting keypass DB.
  7. Make sure ENABLE_JIRA and ENABLE_GOOGLE are set to True. It's turned off by default in an open source setting like this, to prevent awkward behavior for those without the proper creds.
  8. Remove any docker DB you may have from before by checking docker volume ls. If you see something like accounting_postgres_data, do docker volume rm accounting_postgres_data.
  9. Do make dev.build to freshen the container.
  10. Do make dev.up and then make dev.web.logs to see that the service's container is running properly.
  11. Do make migrate.
  12. It may be convenient to have fake data lying around: do touch private.mk and enter these details: 
    users: ## Make all the users we need for testing.
	$(HONCHO_MANAGE) createsuperuser --username me --email me@email.com
	$(HONCHO_MANAGE) createsuperuser --username opencraft --email billing@opencraft.com
	$(HONCHO_MANAGE) generate_account_data --accounts-num 100
    For the first user, please use your JIRA username. You will be using that account for generating the invoice with JIRA worklogs. Run make users. You can do this again later in case you want to delete your DB to start from scratch. Or just use the generate_account_data command directly to make more fake data.
  13. Go in the shell (make shell) and do 
    Site.objects.get_current().update(domain='http://localhost:1786')
  14. Go to http://localhost:1786/admin and login using the first user you created from above.
  15. Go to http://localhost:1786/admin/authentication/user/ and update your user to use your proper full name. This displays on invoices.
  16. Go to http://localhost:1786/admin/account/account/ and add an account for both your user, and for the opencraft user. Up to you if you want it to reflect reality; note though that business names are given priority over full names for display on invoices.
  17. Go to http://localhost:1786/admin/account/hourlyrate/ to make an hourly rate connecting you and the opencraft user, with opencraft being the client.
  18. Go to http://localhost:1786/admin/bank/bankaccount and make a fake bank account. Note that the Identification field is a JSONField -- just enter any arbitrary strings that you'd like to show up on the invoice. Only non-empty strings will actually show. Note also that the invoice only shows the provider's bank account details -- so you can skip doing this for the opencraft user.
  19. Go to http://localhost:1786/admin/invoice/invoice and make a new invoice.
    • Make sure the date-related defaults all make sense: given todays date, the billing start and end are based off of last month, since this is how we assume invoice processing works -- invoices created this month are meant to bill last month's work.
    • Make the provider and client equal to you and OpenCraft, respectively.
    • Upload a signature image (or any image) in the Signature section.
    • Leave checkboxes unchecked.
  20. After saving the invoice, check your repository's media folder and see that the 'signature' you uploaded is present.

JIRA Worklogs Download

  1. Go back to the invoice object you made.
  2. Check the 'Auto download jira worklogs on save' button.
  3. Save! This part can be a bit slow -- there's usually a lot of data being downloaded and processed, and your comptuer's fan might speed up ;p
  4. If you did everything correctly, your system will have correctly downloaded all worklogs for your user from JIRA. Go to http://localhost:1786/admin/invoice/lineitem/ to see that all your worklogs are present.
  5. For some worklog that you have in the past month that's present in the line item list, change the description slightly on JIRA itself.
  6. Re-save the invoice to re-download JIRA worklogs.
  7. Go check the same line item in the admin to ensure that no new item was created, but rather, the existing line item was updated.
  8. You can do the same action with changing the worklog's actual time spent on JIRA, and even if you change the ticket's name! In fact, you can delete, add, update the worklogs as much as you want and the system will ensure you have a perfect sync between the JIRA worklogs for the billing date range and the DB. For example: change the billing end date to many different dates, and re-save. Each time, you'll always have the proper line items in the DB that correspond only to that billing date range, even if you modify the worklogs. Cool!

PDF Generation

  1. Go back to the invoice obejct you made.
  2. Uncheck the 'Auto download jira worklogs on save' button.
  3. Check the 'Auto pdf on save' button.
  4. Save! (This part gets faster the 2nd time around because of heavy DB caching. Actually, that's problematic at some level because the caching may end up giving you false information in the next generated PDF -- for example, if you change the hourly rate right now and re-generate, it would catch the currency change but not the price change.)
  5. Check your media folder to see that an invoice exists.
  6. Inspect the PDF to see that the information all makes sense, including totals and what not. (Ok, you don't have to pull out a calculator, just compare it to your actual invoice from the past month! (that's if your billing start/end dates were for November)).

Google Drive

  1. Go back to the invoice obejct you made.
  2. Do not uncheck the 'Auto pdf on save' button. Google Drive uploading only happens if a PDF is generated first -- the system doesn't go searching in the media folder for candidate PDFs, it gets it directly from the generation code.
  3. Check the 'Auto upload google drive on save'.
  4. Save! (will take however long it takes for the Google Drive API to work)
  5. Go to the 11 folder (for the month of November) in the Accounting Drive's invoice-in folder for 2017. See that your invoice is present and created by crafty.

Play Around

  1. Freebie: play around with this system to see if any normal use cases are broken. I know I didn't implement error handling for basically anything between JIRA worklog downloads / PDF generation / Google Drive uploads -- but see if anything that should work isn't.

Merge request reports