Commit Graph

222 Commits

Author SHA1 Message Date
wowol 700cba3aaf urls: Migrate corporate urls to use modern django pattern. 2020-05-11 16:28:02 -07:00
Anders Kaseorg 7271fb68aa logging: Pass format arguments to unconventionally-named loggers too.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-05-02 16:37:07 -07:00
Vishnu KS 3f69500765 billing: Rename downgrade_realm_for_deactivation to downgrade_now. 2020-04-28 22:05:49 -07:00
Vishnu KS aca80206df billing: Rename change_plan_at_end_of_cycle to change_plan_status. 2020-04-28 22:05:49 -07:00
Anders Kaseorg fead14951c python: Convert assignment type annotations to Python 3.6 style.
This commit was split by tabbott; this piece covers the vast majority
of files in Zulip, but excludes scripts/, tools/, and puppet/ to help
ensure we at least show the right error messages for Xenial systems.

We can likely further refine the remaining pieces with some testing.

Generated by com2ann, with whitespace fixes and various manual fixes
for runtime issues:

-    invoiced_through: Optional[LicenseLedger] = models.ForeignKey(
+    invoiced_through: Optional["LicenseLedger"] = models.ForeignKey(

-_apns_client: Optional[APNsClient] = None
+_apns_client: Optional["APNsClient"] = None

-    notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
-    signup_notifications_stream: Optional[Stream] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
+    notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)
+    signup_notifications_stream: Optional["Stream"] = models.ForeignKey('Stream', related_name='+', null=True, blank=True, on_delete=CASCADE)

-    author: Optional[UserProfile] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE)
+    author: Optional["UserProfile"] = models.ForeignKey('UserProfile', blank=True, null=True, on_delete=CASCADE)

-    bot_owner: Optional[UserProfile] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL)
+    bot_owner: Optional["UserProfile"] = models.ForeignKey('self', null=True, on_delete=models.SET_NULL)

-    default_sending_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
-    default_events_register_stream: Optional[Stream] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
+    default_sending_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)
+    default_events_register_stream: Optional["Stream"] = models.ForeignKey('zerver.Stream', null=True, related_name='+', on_delete=CASCADE)

-descriptors_by_handler_id: Dict[int, ClientDescriptor] = {}
+descriptors_by_handler_id: Dict[int, "ClientDescriptor"] = {}

-worker_classes: Dict[str, Type[QueueProcessingWorker]] = {}
-queues: Dict[str, Dict[str, Type[QueueProcessingWorker]]] = {}
+worker_classes: Dict[str, Type["QueueProcessingWorker"]] = {}
+queues: Dict[str, Dict[str, Type["QueueProcessingWorker"]]] = {}

-AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional[LDAPSearch] = None
+AUTH_LDAP_REVERSE_EMAIL_SEARCH: Optional["LDAPSearch"] = None

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-22 11:02:32 -07:00
Anders Kaseorg f8c95cda51 mypy: Add specific codes to type: ignore annotations.
https://mypy.readthedocs.io/en/stable/error_codes.html

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-22 10:46:33 -07:00
Vishnu KS 94d4bcd4ec billing: Add tests for reupgrade by billing admin. 2020-04-16 19:28:37 -07:00
Vishnu KS 4207f70568 tests: Remove unused stripe fixtures.
These fixtures were added in 4aa2ac1b52.
The fixture name mentions renewal as the test function name. But we
don't have any function called test_renewal in test_stripe file. This
likely means the fixtures were accidentally added. Also, deleting all
fixtures and running --generate-stripe-fixtures don't result in these
fixtures getting generated as well.
2020-04-16 19:28:37 -07:00
Vishnu KS 3f94195d51 billing: Show appropriate message when not on a paid plan. 2020-04-16 19:28:37 -07:00
Vishnu KS dc4d88f6c9 billing: Call stripe_get_customer only if plan is not empty.
Saves an unnecesary call to Stripe if the plan is empty.
2020-04-16 19:28:37 -07:00
Vishnu KS b686bf76a5 billing: Set plan_name in /billing only if last_ledger_entry is not None.
If make_end_of_cycle_updates_if_needed returns None it means the current
plan has been ended. So we should change the value of variable plan_name
only if last_ledger_entry is not None.
2020-04-16 19:28:37 -07:00
Anders Kaseorg c734bbd95d python: Modernize legacy Python 2 syntax with pyupgrade.
Generated by `pyupgrade --py3-plus --keep-percent-format` on all our
Python code except `zthumbor` and `zulip-ec2-configure-interfaces`,
followed by manual indentation fixes.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-04-09 16:43:22 -07:00
Stefan Weil d2fa058cc1
text: Fix some typos (most of them found and fixed by codespell).
Signed-off-by: Stefan Weil <sw@weilnetz.de>
2020-03-27 17:25:56 -07:00
Vishnu KS 96eb1bcd9d tests: Test the case where Customer exists but stripe_customer_id is None. 2020-03-27 16:31:04 -07:00
Vishnu KS 388384e894 tests: Check for object equality instead of type equality.
I don't see any strong reason to do only type equality
when we can do object equality here.
2020-03-27 16:31:04 -07:00
Vishnu KS f2640461d8 billing: Set default value of licensed_used to 0 in /billing.
This will otherwise throw error when the billing page is accessed
after a plan is expired. A proper followup would be to not show
licenses, licenses_used etc if the current plan is on Limited.
2020-03-25 10:54:10 -07:00
Vishnu KS 8b24d40585 billing: Create get_current_plan_by_realm helper function. 2020-03-25 10:54:10 -07:00
Vishnu KS 83da23c0d4 billing: Rename get_current_plan to get_current_plan_by_customer.
Also add tests.
2020-03-25 10:54:10 -07:00
Vishnu KS 9a2c64f3f4 billing: Create get_customer_by_realm helper function. 2020-03-25 10:54:10 -07:00
Vishnu KS f8ddab58ba billing: Downgrade plan to Limited during realm deactivation.
The realm would be instantly downgraded to Limited plan when
deactivated. Any extra users that were added in the final month
would not be charged.
2020-03-25 10:54:10 -07:00
Steve Howell 1306239c16 tests: Use email/delivery_email more explicitly.
We try to use the correct variation of `email`
or `delivery_email`, even though in some
databases they are the same.

(To find the differences, I temporarily hacked
populate_db to use different values for email
and delivery_email, and reduced email visibility
in the zulip realm to admins only.)

In places where we want the "normal" realm
behavior of showing emails (and having `email`
be the same as `delivery_email`), we use
the new `reset_emails_in_zulip_realm` helper.

A couple random things:

    - I fixed any error messages that were leaking
      the wrong email

    - a test that claimed to rely on the order
      of emails no longer does (we sort user_ids
      instead)

    - we now use user_ids in some place where we used
      to use emails

    - for IRC mirrors I just punted and used
      `reset_emails_in_zulip_realm` in most places

    - for MIT-related tests, I didn't fix email
      vs. delivery_email unless it was obvious

I also explicitly reset the realm to a "normal"
realm for a couple tests that I frankly just didn't
have the energy to debug.  (Also, we do want some
coverage on the normal case, even though it is
"easier" for tests to pass if you mix up `email`
and `delivery_email`.)

In particular, I just reset data for the analytics
and corporate tests.
2020-03-19 16:04:03 -07:00
Vishnu KS a48845c8fb tests: Create new test mode for generating stripe fixtures. 2020-03-17 16:46:56 -07:00
Steve Howell 1b16693526 tests: Limit email-based logins.
We now have this API...

If you really just need to log in
and not do anything with the actual
user:

    self.login('hamlet')

If you're gonna use the user in the
rest of the test:

    hamlet = self.example_user('hamlet')
    self.login_user(hamlet)

If you are specifically testing
email/password logins (used only in 4 places):

    self.login_by_email(email, password)

And for failures uses this (used twice):

    self.assert_login_failure(email)
2020-03-11 17:10:22 -07:00
Mateusz Mandera c0822739d0 corporate: Allow customer to go through /upgrade/ if their plan ended.
This specifically fixes an issue where a customer chose monthly payment
plan instead of annual, getting stuck with the higher monthly payments.
With this change, the Customer plan can be set to ENDED and they can go
through /upgrade/ again and choose annual payments.
A proper follow-up should be to make it possible to change between
monthly/annual on the /billing/ page.
2020-02-13 17:40:04 -08:00
rht 41e3db81be dependencies: Upgrade to Django 2.2.10.
Django 2.2.x is the next LTS release after Django 1.11.x; I expect
we'll be on it for a while, as Django 3.x won't have an LTS release
series out for a while.

Because of upstream API changes in Django, this commit includes
several changes beyond requirements and:

* urls: django.urls.resolvers.RegexURLPattern has been replaced by
  django.urls.resolvers.URLPattern; affects OpenAPI code and related
  features which re-parse Django's internals.
  https://code.djangoproject.com/ticket/28593
* test_runner: Change number to suffix. Django changed the name in this
  ticket: https://code.djangoproject.com/ticket/28578
* Delete now-unnecessary SameSite cookie code (it's now the default).
* forms: urlsafe_base64_encode returns string in Django 2.2.
  https://docs.djangoproject.com/en/2.2/ref/utils/#django.utils.http.urlsafe_base64_encode
* upload: Django's File.size property replaces _get_size().
  https://docs.djangoproject.com/en/2.2/_modules/django/core/files/base/
* process_queue: Migrate to new autoreload API.
* test_messages: Add an extra query caused by .refresh_from_db() losing
  the .select_related() on the Realm object.
* session: Sync SessionHostDomainMiddleware with Django 2.2.

There's a lot more we can do to take advantage of the new release;
this is tracked in #11341.

Many changes by Tim Abbott, Umair Waheed, and Mateusz Mandera squashed
are squashed into this commit.

Fixes #10835.
2020-02-13 16:27:26 -08:00
Tim Abbott 8e7ce7cc79 python: Sort migrations/management command imports with isort.
This is a preparatory commit for using isort for sorting all of our
imports, merging changes to files where we can easily review the
changes as something we're happy with.

These are also files with relatively little active development, which
means we don't expect much merge conflict risk from these changes.
2020-01-14 13:07:47 -08:00
Tim Abbott f83871c182 corporate: Consistently use delivery_email for billing.
This is yet another corner case where EMAIL_ADDRESS_VISIBILITY_ADMINS
broke existing code.
2019-11-18 17:02:57 -08:00
Anders Kaseorg 93b1c3d94b settings: Extract config file functions to a module.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 12:38:45 -08:00
Anders Kaseorg cafac83676 request: Tighten type checking on REQ.
Then, find and fix a predictable number of previous misuses.

With a small change by tabbott to preserve backwards compatibility for
sending `yes` for the `forged` field.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-11-13 12:35:55 -08:00
Rishi Gupta 0b39263ec0 billing: Rename get_seat_count to get_latest_seat_count.
This will help as our billing system becomes more async to accommodate
on-prem billing.
2019-10-21 16:06:18 -07:00
Mateusz Mandera bbf2474bd0 tests: setUp overrides should call super().setUp().
MigrationsTestCase is intentionally omitted from this, since migrations
tests are different in their nature and so whatever setUp()
ZulipTestCase may do in the future, MigrationsTestCase may not
necessarily want to replicate.
2019-10-19 17:27:01 -07:00
Rishi Gupta e10361a832 models: Replace is_guest and is_realm_admin with UserProfile.role.
This new data model will be more extensible for future work on
features like a primary administrator.
2019-10-06 16:24:37 -07:00
Anders Kaseorg 7494f1600c templates: Move page_params from an inline script to the <body> dataset.
This sidesteps tricky escaping issues, and will make it easier to
build a strict Content-Security-Policy.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-09-17 16:06:33 -07:00
Vishnu Ks d472d30c28 requirements: Upgrade stripe pip package from 2.21.0 to 2.35.0. 2019-08-19 11:09:33 -07:00
Vishnu KS 6c8d837401 stripe: Don't check value of realm_id directly in tests.
Value of realm_id varies from one development environment
to another from time to time. So we will just check whether
realm_id is a number instead.
2019-08-19 11:09:33 -07:00
Mateusz Mandera d3d62f99e4 tests: Adjust to new changes in populate_db. 2019-07-31 16:01:37 -07:00
Wyatt Hoodes 45e37d16c3 corporate/views: Fix typing errors. 2019-07-29 15:23:52 -07:00
Greg Price de66b3b0f1 corporate: Add a /jobs page, with job posts for mobile/fullstack.
With various edits from Rishi and Tim.
2019-05-22 22:20:58 -07:00
Anders Kaseorg 9a9de156c3 lint: Fix calls to _() on computed strings.
Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2019-04-23 15:23:03 -07:00
Rishi Gupta 1a7a449572 billing: Add backend support for downgrading. 2019-04-11 23:27:42 -07:00
Rishi Gupta babaaf82fe billing: Update json endpoints test to allow more flexibility.
Slight loss in functionality, but doing the mocks correctly for every view
will be more work than the test is worth.
2019-04-11 23:20:29 -07:00
Rishi Gupta a529080f01 billing: Rename add_plan_renewal_to_license_ledger_if_needed. 2019-04-11 23:20:29 -07:00
Rishi Gupta 03d21c6317 billing: Refactor renewal_amount to return 0 instead of None. 2019-04-11 23:20:29 -07:00
Rishi Gupta 447ae77b24 billing: Add another live CustomerPlan.status.
Only CustomerPlan.ACTIVE is being used in the code / in production, so don't
need a db migration to move the other CustomerPlan.status constants around.
2019-04-11 23:20:29 -07:00
Rishi Gupta 4430f78aa3 billing: Rename next_renewal_date to start_of_next_billing_cycle. 2019-04-11 23:20:29 -07:00
Rishi Gupta 2270d4d192 billing: Try paying invoices when user puts a new card on file.
Previously, when users got a "payment failed" email from Stripe (e.g. if
their card failed on renewal), they would enter in a new card on
/billing#payment-method, and wouldn't find out if the card worked till
Stripe retried the payment 4 days later.
2019-04-05 13:36:03 -07:00
Rishi Gupta 5d970cc09b billing: Upgrade to Stripe API version 2019-03-14.
No breaking changes.
2019-04-05 11:24:45 -07:00
Rishi Gupta 28a586e502 billing: Add default value for event_time in invoice_plans_as_needed. 2019-04-05 11:24:45 -07:00
Vishnu Ks ca1276961d billing: Create get_discount_for_realm function. 2019-03-11 12:01:11 -07:00
Anders Kaseorg f5197518a9 analytics/zilencer/zproject: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:31:45 -08:00
Anders Kaseorg 4e21cc0152 views: Remove unused imports.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
2019-02-02 17:23:43 -08:00
Rishi Gupta 8f7e38c559 billing: Remove deprecated billing tables and fields from corporate/models. 2019-01-31 15:12:43 -08:00
Rishi Gupta 55179f913c billing: Remove references to Customer.has_billing_relationship. 2019-01-31 15:12:43 -08:00
Vishnu Ks 18ffec626f billing: Use -- instead of : in fixture files.
It's not safe to use ":" in filenames.

Fixes #11320
2019-01-31 12:26:00 -08:00
Rishi Gupta a37558b021 billing: Update get_seat_count to incorporate guests. 2019-01-30 10:35:01 -08:00
Rishi Gupta db6e6472b1 billing: Set idempotency key to None when running test suite. 2019-01-30 10:35:01 -08:00
Rishi Gupta 60f2213451 billing: Change seat count to 6 in tests.
We were about to make it 7 anyway by changing the definition of
get_seat_count. Hopefully 6 will be a bit more future proof.
2019-01-30 10:35:01 -08:00
Rishi Gupta 1b0b72c827 billing: Change upgrade errors from info to warning. 2019-01-29 07:46:52 -08:00
Rishi Gupta 81745ba171 billing: Show licenses_used and better renewal information on /billing. 2019-01-29 07:38:24 -08:00
Rishi Gupta 4ccbeeb4a9 billing: Add an event_time argument to renewal_amount. 2019-01-29 07:37:21 -08:00
Rishi Gupta a5324b6ea7 billing: Add a test for a race condition in process_initial_upgrade. 2019-01-29 07:01:31 -08:00
Rishi Gupta 9018ef5175 billing: Do not require a Stripe account to store Customer discounts. 2019-01-29 06:09:11 -08:00
Rishi Gupta 21b51ba8d1 billing: Remove mix option from backend. 2019-01-29 06:09:11 -08:00
Rishi Gupta 4aa2ac1b52 billing: Add functions for automatic invoicing of plans. 2019-01-28 16:04:11 -08:00
Rishi Gupta 421cda0e34 billing: Adding invoicing fields to CustomerPlan. 2019-01-28 16:04:11 -08:00
Rishi Gupta fe280fc38c billing: Create StripeTestCase. 2019-01-28 16:04:11 -08:00
Rishi Gupta 83a7595feb billing: Update next_renewal_date to take an event_time.
Also changes a < into a <= in add_plan_renewal_to_license_ledger_if_needed.
2019-01-28 16:04:11 -08:00
Rishi Gupta 7c11fe819a billing: Update license ledger when users are added and removed. 2019-01-28 16:04:11 -08:00
Rishi Gupta 03c71dad17 billing: Fix misuses of CustomerPlan.discount. 2019-01-28 16:04:11 -08:00
Rishi Gupta ad7a7b246e billing: Move license management from CustomerPlan into its own table. 2019-01-19 16:00:44 +05:30
Rishi Gupta 7958ac96a8 billing: Use CASCADE instead of models.CASCADE in models.py.
models.CASCADE will make lines too long in models we're about to add.
2019-01-18 11:46:17 +05:30
Rishi Gupta 112eb01728 billing: Remove PLAN_NAMES and other legacy code. 2019-01-18 11:46:17 +05:30
Rishi Gupta 955dfc6397 billing: Remove references to Realm.has_seat_based_plan. 2018-12-24 10:27:56 -08:00
Rishi Gupta 7c38300949 billing: Add info line to logger for failed upgrades. 2018-12-24 10:27:56 -08:00
Rishi Gupta 653416ab23 billing: Update /billing to work with new subscription model. 2018-12-23 00:18:21 -08:00
Rishi Gupta e7220fd71f billing: Do subscription management in-house instead of with Stripe Billing.
This is a major rewrite of the billing system. It moves subscription
information off of stripe Subscriptions and into a local CustomerPlan
table.

To keep this manageable, it leaves several things unimplemented
(downgrading, etc), and a variety of other TODOs in the code. There are also
some known regressions, e.g. error-handling on /upgrade is broken.
2018-12-22 13:39:30 -08:00
Rishi Gupta 5633049292 billing: Restructure validation of upgrade parameters. 2018-12-22 13:30:15 -08:00
Rishi Gupta b4a28f3147 billing: Rename seat_count to licenses where appropriate. 2018-12-22 13:30:15 -08:00
Rishi Gupta 7ab1406962 billing: Prepare for moving Plan to CustomerPlan.billing_schedule. 2018-12-22 13:30:15 -08:00
Vishnu Ks 8176d112fe billing: Add frontend for license-based billing system. 2018-12-22 13:30:15 -08:00
Rishi Gupta 7b5d15d254 billing: Move discount to local Customer object.
A lot of the seemingly unrelated test fixture changes are because we're
removing a query to stripe in the upgrade path, in cases when the user's
realm has an existing Customer object.
2018-12-22 13:30:15 -08:00
Rishi Gupta 8ec91fc42d billing: Remove process_downgrade. 2018-12-22 13:30:15 -08:00
Rishi Gupta 84a31ca800 billing: Remove BillingProcessor.
Leaving the model in place, so that we can do the database migration by
hand.
2018-12-22 13:30:15 -08:00
Vishnu Ks 0fd6ff722b billing: Migrate /upgrade endpoint to JSON.
The fixture changes are because self.upgrade formerly used to cause a page load
of /billing, which in turn calls Customer.retrieve.

If we ran the full test suite with GENERATE_STRIPE_FIXTURES=True, we would
likely see several more Customer.retrieve.N.json's being deleted. But
keeping them there for now to keep the diff small.
2018-12-13 17:01:12 -08:00
Rishi Gupta b245c661da billing: Change do_change_plan_type to take a realm instead of a user.
More often than not, changes in plan type are not directly due to user
action.
2018-12-13 13:26:24 -08:00
Vishnu Ks a8113ae296 billing: Remove dont_mock argument from mock_stripe. 2018-12-04 12:19:57 -08:00
Rishi Gupta c21a05a3b1 billing: Zero out timestamps that aren't being tested.
f52e9d1 ended up not going far enough. Keeping f52e9d1 in place in case we
ever want to go back to that sort of solution.

Also removes the keep argument from test_billing_quantity_changes_end_to_end,
since that test is actually testing the arguments to
stripe.Subscription.save(), not what is returned by Stripe.
2018-12-03 16:25:21 -08:00
Rishi Gupta 2453ab2a21 billing: Normalize a few more fields in normalize_fixture_data. 2018-12-03 16:24:16 -08:00
Rishi Gupta 84f7a779c4 billing: Take advantage of re match notation in normalize_fixture_data.
No change in behavior; just visual code cleanup.
2018-12-03 16:24:16 -08:00
Rishi Gupta f52e9d143b billing: Further normalize timestamps in tests.
Hopefully this will make it easier still to look at diffs with
GENERATE_STRIPE_FIXTURES = True.
2018-12-03 16:24:16 -08:00
Vishnu Ks 24917f2e9b billing: Maintain a global list of stripe functions to mock.
Looks like payment_method_string was missing a Customer.save before. Not
sure how it was passing tests in that case.
2018-12-03 11:38:54 -08:00
Vishnu Ks 0191bb593a billing: Make test functions accept *mocks instead of separate args. 2018-12-02 13:42:38 -08:00
Vishnu Ks 48618adf13 billing: Normalize several additional keys in mock_stripe.
Reran every test with GENERATE_STRIPE_FIXTURES = True, which also caused a
few fixtures to get updates unrelated to these changes (likely due to API
updates that hadn't been previously applied).
2018-12-02 13:32:19 -08:00
Rishi Gupta c9c842ae2b billing: Add randomness to idempotency key while testing.
Makes it easier to test or regenerate fixtures for
test_billing_quantity_changes_end_to_end with generate=True.
2018-11-30 23:14:52 -08:00
Rishi Gupta c59da02936 billing: Reflect discounts on /upgrade. 2018-11-30 11:19:07 -08:00
Rishi Gupta 16b7737da3 billing: Fix enforcement of seat_count in tests. 2018-11-30 11:19:07 -08:00
Rishi Gupta 5a47ac96de billing: Fix test flake in test_payment_method_string. 2018-11-30 11:19:07 -08:00
Rishi Gupta 1ba037201c billing: Move all price computations into billing.js. 2018-11-30 11:19:07 -08:00
Rishi Gupta 5fdc2e5161 billing: Ensure seat_count is always 8 in billing tests. 2018-11-28 23:08:27 -08:00
Rishi Gupta f3ecb05762 billing: Rename stripeToken to stripe_token.
When we started the billing system we started by following conventions used
in the Stripe documentation, but in hindsight it makes more sense to follow
conventions used in the Zulip codebase.
2018-11-28 22:29:29 -08:00