Earlier, at few places in test_stripe we were doing
incorrect time travel. For example:
A plan was invoiced till self.next_year & while checking
the billing page, we were using the self.now datetime.
To correctly check the billing page state after plan is
invoiced, we should use a datetime greater than or equal
to the time at which plan was invoiced.
This commit fixes such logically incorrect time-travel.
While creating a LicenseLedger entry in 'create_customer_and_plan',
we should set the 'event_time' to the same time at which the plan
is created.
Earlier, the 'event_time' for ledger entry & 'billing_cycle_anchor'
of the plan were set to different values, which is not the
correct behavior.
Earlier, the code block to calculate additional licenses charge
raised assertion error when processing a fixed-price plan.
The code block shouldn't be executed for fixed-price plans as
we don't charge customers on fixed-price plans for additional
licenses.
Earlier, the 'self.on_paid_plan()' check was verifying if the
billing_session/customer is on paid plan and not the plan we
are processing.
This resulted in a bug. While processing a legacy plan, a customer
switches from legacy plan to a paid plan resulting in the
'self.on_paid_plan()' check returning True.
It leads to invoicing legacy plan which shouldn't happen.
The fix is to check if the plan we are processing is paid or not
instead of the remote_realm/remote_server plan_type.
It's best for these to just be consistent. Therefore:
1. The .../not-configured/ error page endpoint should be restricted to
.has_billing_access users only.
2. For consistency, self_hosting_auth_view_common is tweaked to also do
the .has_billing_access check as the first thing, to avoid revealing
configuration information via its redirect/error-handling behavior.
The revealed configuration information seems super harmless, but it's
simpler to not have to worry about it and just be consistent.
Just shows a config error page if the bouncer is not enabled. Uses a new
endpoint for this so that it can work nicely for both browser and
desktop app clients.
It's necessary, because the desktop app expects to get a json response
with either an error or billing_access_url to redirect to. Showing a
nice config error page can't be done via the json error mechanism, so
instead we just serve a redirect to the new error page, which the app
will open in the browser in a new window or tab.
The remote support view now returns results for deactivated remote
servers with those results sorted to the end and formatted to
visually stand out.
Forms to change sponsorship and discount fields on the customer
for the remote server or realm are not shown, but the data stored
on the customer object is shown, including any sponsorship request
information (if the customer had a sponsorship request pending when
it was deactivated).
Forms to schedule a plan are also not shown for deactivated servers
and their associated remote realms.
Forms and information for any current plan or scheduled plan, for
either the deactivated remote server or its associated remote
realms, are shown so that support staff can update those plans if
necessary.
Earlier, the 'next_invoice_date', 'invoiced_through', and
'invoicing_status' fields in 'do_change_schedule_after_free_trial'
were not set correctly.
It resulted in invoicing the ENDED plan and the ledger we create
in this function.
Multiple invoices were created depending on the number of times the
billing frequency was changed for customers who started free-trial
& changed their billing frequency.
This commit sets those fields correctly leading to create only one
invoice.
As a follow-up to commit d66b7ad853, where we send internal emails
when an active paid plan is on a locally deleted remote realm, we
need search queries in the remote support view to return results
for these deactivated remote realms, instead of excluding them.
This is a prep commit which replaces the 'invoice_overdue'
and 'reminder_to_review_plan' email templates with
'internal_billing_notice'.
This will help us to use the same template as we plan to
send an email to sales when a remote realm with paid plan
attached is locally deleted.
Because the remote support page now supports searching by UUID,
the support URL for remote billing entities, which is used for
sponsorship request emails and overdue invoice emails, can now
use the remote server or realm UUID.
Adds the remote realm UUID to the remote support view information.
Earlier, for fixed-price plans we were showing the generic
next payment info on billing page which stated that plan
will automatically renew on end_date. It is no longer correct
for fixed-price plans.
This commit fixes the next payment info for fixed-price plans.
When the next_billing_cycle is the end_date, we inform the customer
that their plan will end on end_date and zulip sales will contact
them a couple of month ago before the end_date for renewal.
Earlier, the 'renewal_amount' context variable passed to the billing
template was always a non-empty string which is a truthy value.
The expected behavior is that 'renewal_amount' should be falsy value
if the renewal_cents is 0. This commit fixes the incorrect behavior
and returns None in this case.
Adds the information returned by get_push_status_for_remote_request
for remote billing users to the support page. Note that getting
the current push status data will result in some duplicate database
queries (getting customer, plan, current billed users, next billing
cycle) when generating the remote support view.
The previous logic incorrectly used the server-level number of users
even when a (presumably smaller) realm-level count was available.
Fixes a bug introduced in 2e1ed4431a.
Instead of querying the database twice for the user counts and
for the currently used licenses, we know pass that information
to the function where we populate the plan data for the remote
server or realm.
Only get the customer information once from the database.
Extract logic for any next plan (either scheduled or an offer) data
into a separate function and do not hit database twice when getting
the number of licenses at the next plan renewal.
Removes prefetch of remote realms on the remote server query
because, by excluding the system bot realm when this is referenced,
we end up hitting the database again.
Strips whitespace from query string and makes a few small updates
for readability.
We no longer want to migrate Legacy plans from server to realms, since
Legacy plans are not really a thing in the original sense anymore, since
February 15th.
Now they're just a tool to give temporary extensions of access to the
push notification service for users, when needed. And as such, it makes
no sense to migrate like that.
The remaining code in this function is for migrating (any) plan from the
server object to the realm object, if the server has just a single
realm.
For fixed-price plans, we send a reminder email to
sales@zulip.com, 2 months before the end date, to review
the pricing and configure a new fixed-price plan accordingly.
Earlier, we were not configuring the end_date while creating
a fixed-price plan which would result in the automatic renewal
of the plan at the end of billing cycle.
Our plan is to re-evaluate the pricing when we are close to the
end date, then:
1. Schedule a new fixed-price plan with the updated or same price.
2. or Don't do anything - the plan will end on the end_date.
Now, we add an end_date of 1 year from the start date when
creating a fixed-price plan.
This is a prep commit which extracts the write_to_audit_log
code block in `update_end_date_of_current_plan` into a
function named 'write_to_audit_log_plan_property_changed'.
This would be helpful to avoid repetition as we plan to
include another such block when the plan's property
reminder_to_review_plan_email_sent is changed.
The logic in the case where there's only one realm and the function
tries to migrate the server's plan to it, had two main unhandled edge
cases that would throw exceptions:
1.
```
remote_realm = RemoteRealm.objects.get(
uuid=realm_uuids[0], plan_type=RemoteRealm.PLAN_TYPE_SELF_MANAGED
)
```
This could throw an exception if the RemoteRealm exists, but has an
active e.g. Legacy plan. Then there'd be no object matching the
plan_type in the query, raising RemoteRealm.DoesNotExist.
2. If the RemoteRealm had e.g. a Legacy plan in the past, that's now
expired, then it'd have a Customer object. Meaning that the attempt
to move the server's customer to the realm:
`server_plan.customer = remote_realm_customer`
would trigger an IntegrityError since a RemoteRealm can't have two
Customer objects.
In simple cases the situation in (2) can still be easily migrated, by
moving the plan from the server's customer to the realm's customer.
This is kind of too specific, allowing testing for only one single error
when accessing signed_auth_url. Instead, this should use a general
pattern, which will allow other tests to use this to assert other kinds
of error responses that may be returned.
It doesn't make sense to run a loop over "all" query results, when those
results are just two, and each of them has its own distinct asserts.
That for loop is there probably due to copying the structure of the
earlier test_transfer_legacy_plan_from_server_to_all_realms test, for
which the loop does make sense.
In 'invoice_plan()', the primary way to not create
an invoice for a plan is to not have any new ledger entry.
This commit adds a 'self.on_paid_plan()' check which is
an extra layer of defense to avoid creating any invoices
for customers not on paid plan. It saves a DB query too.
RemoteRealm customer takes precedence over RemoteServer
in general. But if an inactive plan is associated with
RemoteRealm and an active plan with RemoteServer, the
ACTIVE plan takes precendence.
Co-authored-by: Prakhar Pratyush <prakhar@zulip.com>
Instead of displaying the end of the day interval for the latest
count stat update for push notifications forwarded on the bouncer,
we display the start of the day interval and format it as a date
instead of a date and time.
In commit 230294c, the logic for creating the support links in the
remote activity chart were broken. Updates the constants so that
the links have the correct server information for the search query.
Earlier, during invoicing we used to send a mail to
sales@zulip.com when the last_audit_log_update was
at least one day ago.
There was an assertion that last_audit_log_update
can't be None which is incorrect as such customers exist.
This commit extends the behaviour to send an invoice
overdue email even if the data was never uploaded.
In cases where a support admin approves a sponsorship for a remote
server without any billing users having been created, note that no
emails were sent in the success message on the support page.
We already do this for remote realms and should be an infrequent
case as ideally most remote servers will submit official
sponsorship requests.
Earlier, on extending end_date for legacy plans, next_invoice_date
was not extended which resulted in invoice_plan() run for such
legacy plans before the end_date.
The intended behaviour is that the legacy plan should be invoiced
only once on the end_date to downgrade or switch to a new tier.
This commit adds logic to update next_invoice_date when end_date
is extended via /support.
Expands the main query for remote servers to get the audit log
event datetime for when the server was created/registered.
The remote realm object has a field for when the remote realm
was created on the remote server.
Adds a link on the upgrade and billing pages that opens a stripe
billing portal for the customer to update their name and address
that will appear on invoices and receipts.
On the billing page, updating the credit card information will
no longer update the customer billing address, since they can
now do this directly through the billing portal. To be consistent
with the credit card form on the upgrade page, we still require
inputting a billing address for the card.
Note that, once an invoice is paid/complete, then changes to the
customer's name and address will not be applied to those invoices.
Set the name on the stripe customer object to the name on the
credit card when one is initially attached to a customer.
Prep commit to adding a stripe billing portal so that billing
admins will be able to update both the name and address that
appears on their stripe generated invoices and receipts.
Instead of charging the customer using the attached payment
method and then creating the invoice, we create an invoice and
force an immediate payment for the invoice via the attached
payment method.
Earlier, we were not setting `free_trial` = False for legacy
customers in `do_upgrade`.
This lead to a bug where customers were upgraded even before the
payment was complete. 'process_initial_upgrade' was always getting
called instead of 'setup_upgrade_payment_intent_and_charge'.
This commit fixes the incorrect behaviour.
Adds a count for mobile users registered for the remote server
with a RemotePushDeviceToken that does not have an associated
remote realm, which was a recently added field.
If the remote server is pre-8.0 and does not have remote realms,
then only the total mobile user count is displayed, as the count
for uncatagorized mobile users would be equal to the total mobile
user count.
This is useful in the support view in case the audit log data is
stale and user counts are not updated for billing.
Also, renames formatting function for optional datetimes that is
used in the support and activity views/charts. And instead of
showing these datetime strings in the eastern US timezone, we
now show and label them as UTC.
This will require customers to include an address when setting
up, or updating, the credit card information for their account.
The billing address for the card will also be saved as the
billing address for the stripe customer object.
The customer object billing address appears on the invoices
that are generated by stripe.
Adds a column with the percentage rate that the remote server
or realm is paying on the displayed plan.
We display 0% for community plans that are 100% sponsored.
For legacy plans or plans with a scheduled downgrade, we
display a placeholder, "---". Otherwise, the value is
calculated from the CustomerPlan discount field.
Earlier, when a fixed-price plan for a customer with
no current plan was configured via /support, the next plan
info was missing on support page.
It was because we were considering next plan only if the
customer had a current plan.
This commit fixes the incorrect behaviour.
If a plan is already on "basic" or "business" plan and wants to
switch to a fixed-price "basic" / "business" plan, then it is
necessary that the current plan should have an end date configured.
Earlier, we could update the end_date only if the current plan
already had an end_date set.
This commit makes it possible to always show the option to
set or update end_date.
While configuring fixed price offer via /support, the
status for fixed price plan & offer are not the same.
Updates the code to not set it initially & then overwrite,
instead set it while creating the plan / offer.
Also, makes small updates to `next_plan_forms_support.html`.
Removes unneeded "btn" and "btn-default" classes, and updates
the placeholder text for the input as not marked for translation.
Also, renames `ad_hoc_query.html` to `activity_table.html`,
`realm_summary_table.html` to `installation_activity_table.html`,
and `activity_details_template.html` to `activity.html`.
Removes the style attribute in the installation activity template
and uses a CSS class, "installation-activity-header", to center the
h3 and p tags instead. This removes an exception from the custom
lint check.
View functions in `analytics/views/support.py` are moved to
`corporate/views/support.py`.
Shared activity functions in `analytics/views/activity_common.py`
are moved to `corporate/lib/activity.py`, which was also renamed
from `corporate/lib/analytics.py`.
Updates the total row for the installation and remote activity
charts to be in the table footer. Makes the footer class sticky
to the bottom of the view so that it is always visible on the
chart.
Also, updates the installation activity column for revenue to
be formatted as a dollar string, since this formatting was being
applied in the updated total row.
To estimate the annual recurring revenue for remote server and
remote realm CustomerPlans, we prefetch the current license
ledger as part of the CustomerPlan query.
Also adds a select related to the remote realm audit log query
so that we don't go to the database for the remote realm ID.
With the test added in the previous commit, the query count for
the remote activity view goes from 27 to 11, as we are no longer
hitting the database multiple times for every current plan or
for every remote realm with audit log data.
Refactors get_customer_plan_renewal_amount so that a license
ledger is always passed and make_end_of_cycle_updates_if_needed
does not need to be called.
Earlier, the next_invoicing_date and invoicing_status
for new plan weren't set correctly, resulting in the
scheduled switching of legacy plan to a new plan not
working as expected.
This commit fixes the incorrect behaviour.
Earlier, in 'migrate_customer_to_legacy_plan`, we set
'next_invoice_date' to None for legacy plans.
This will result in legacy plans not getting invoiced.
We need to invoice legacy plans on their end date to
either downgrade them or switch to a new plan.
We set next_invoice_date for legacy plans to end_date.
Instead of showing the next invoice date for the plan, show the
date for the next billing cycle start (e.g. the next plan renewal
charge date), except for plans currently on a free trial.
For plans on a free trial, the next plan renewal date will be when
the free trial ends, which is stored as the next invoice date on
the plan.
Instead of querying the database for every remote server and realm
in the remote activity chart, we now get the server and realm data
for the installation in two queries.
Adds columns for remote realm ID, name and organization type. If
a remote server has remote realms attached that are not marked
as deactivated by the remote server, then there will be a row in
the chart for each remote realm (which duplicates some remote
server data).
Updates the plan data, revenue and user counts to be for the realm
if present and otherwise for the server.
Updates the user counts to be total users and guest users, instead
of non guest and guest users.
The total row for mobile data (users and pushes forwarded) sums
each remote server's data once, so while the column duplicates
data, the total row should be an accurate total for the installation.
Adds 5 queries to the remote activity page test. One is for the
additional query for the remote realm plans. The other four are
getting the remote realm object and then the user count data for
the two remote realms in the test.
When you click "Plan management", the desktop app opens
/self-hosted-billing/ in your browser immediately. So that works badly
if you're already logged into another account in the browser, since that
session will be used and it may be for a different user account than in
the desktop app, causing unintended behavior.
The solution is to replace the on click behavior for "Plan management"
in the desktop app case, to instead make a request to a new endpoint
/json/self-hosted-billing, which provides the billing access url in a
json response. The desktop app takes that URL and window.open()s it (in
the browser). And so a remote billing session for the intended user will
be obtained.
Now that a customer discount may require a particular plan tier to
be applied, update the billing code to check the plan tier when
getting the customer default_discount field/information for a new
plan.
For billing schedule changes and displaying billing information for
current plans, we explicitly use the discount set on the current,
active plan and do not check the customer object for these actions.
This will be used to set a required plan tier value to be used with
the default discount that is set on the Customer object or with a
fixed price set on a CustomerPlan object.
This prep commit adds logic to calculate discount based
on flat_discount and flat_discounted_months. Creates
a stripe invoice item for the discount.
This will be used by remote realm/server billing system
while invoicing via cron job.
This commit renames the variable 'realm_user_count' to
'server_user_count' in 'test_upgrade_user_to_monthly_basic_plan'.
The variable was incorrectly named earlier as it stores
the user count of the whole server.
This was a bug from 4715a058b0 where this
was just incorrectly called. get_realms_info_for_push_bouncer() is a
function meant to be called on a self-hosted server - and this
handle_... call happens on the bouncer. Therefore this returns all
zulipchat realms in product.
With the way, handle_... is being called right now, there's no reason
for it to have an argument for passing a list of realms. It should just
fetch the relevant RemoteRealm entries by itself, given the server arg.
The bug was that a user could do the first part of the flow twice,
receiving two confirmation links, before finishing signup. Then they
could use the first link, followed by the second, which would case an
IntegrityError due to trying to create the RemoteRealmBillingUser
for the second time.
When the second link gets clicked, we should just transparently redirect
the user further into the flow so that they can proceed.
Earlier, in process_initial_upgrade, the flat_discount value
wasn't converted into dollars when specified in the invoice
description, resulting in showing the incorrect value of $2000
as a discount.
This commit converts the value in cents to dollars and adds tests
to verify the invoice generated.
Updates the HTML input field to have a min of 0, max of 99.99 and
allow increments of 0.01.
Also, use format_discount_percentage for displaying the customer
default discount in the support form.
For self hosted basic plan, we need to allow customers to subscribe
without purchasing 10 licenses and also we need to allow customer
take fully use the available discount so that if the add more
users in the future, the full discount was already applied.
To fix above, we set minimum user count to the least number
of licenses we require for the charge to be positive after applying
the complete discount.
We return expected_end_timestamp as "None" for the plans to be
downgraded if number of users is not more than MAX_USERS_WITHOUT_PLAN
since they will be downgraded to self-managed plan and would
have push notifications enabled.
Earlier, the 'handle_customer_migration_from_server_to_realms'
function was called during the send analytics step.
It resulted in an error for customers having multiple Zulip servers,
one for testing and the others for not-testing, sharing a
push bouncer registration.
The migration step when run in a test instance caused customers to
have their legacy plan migrated to a test realm, resulting in them
losing their legacy plan.
This commit moves the migration step to run during plan management
login step. This reduces the chances of losing legacy
plan as we expect them to only verify that 8.0 upgrade works and
not bother trying to login to plan management from their test instance.
Adds a support action for updating the minimum licenses on a
customer object once a default discount has also been set.
In the case that the current billing entity has a current active
plan or a scheduled upgrade to a new plan, then the minimum
licenses will not be updated.
Previously, the message string was sent as a success response to
the context, which could have been confusing or ignored when shown
in the support admin view.
- Make `self.write_to_audit_log` support a `background_update:
bool=False` parameter that can be passed when code that might have an
acting user happens to trigger a background update.
- Make `make_end_of_cycle_updates_if_needed` pass that parameter for its
direct audit log writes.
- Audit code that `make_end_of_cycle_updates_if_needed` calls and make
sure those write audit logs this way too.
- Pass the user in the `billing_page` code that had to avoid it as a
workaround:
```
# BUG: This should pass the acting_user; this is just working
# around that make_end_of_cycle_updates_if_needed doesn't do audit
# logging not using the session user properly.
billing_session = RealmBillingSession(user=None, realm=user.realm)
```
Creates some reusable helper functions and adds remote realms to
the search results that are checked, which gives coverage for the
remote realm user counts in the support view.
Also fixes formatting for per license price and moves the billing
schedule to be above this line so that it's clearer the per license
price is based on the billing schedule.
This moves the function which computes can_push and
expected_end_timestamp outside RemoteRealmBillingSession
because we might use this function for RemoteZulipServer
as well and also renames it.
This prep commit adds a 'billing_session' field to StripeTest class.
Creates 'client_billing_get', 'client_billing_post', and
'client_billing_patch' helper functions.
This will help in reusing code for RemoteRealm and
RemoteZulipServer end-to-end tests.
The call to 'get_billable_licenses_for_customer' during the
'sync_license_ledger_if_needed' step should use the audit_log's
event_time while calculating 'current_count_for_billed_licenses'.
Earlier, it used timezone_now(), resulting in the latest user count
recorded corresponding to each audit log.
If server has plan, deny login for realm.
If realm has plan, deny login for server.
Co-authored-by: Aman Agrawal <amanagr@zulip.com>
Co-authored-by: Alya Abbott <alya@zulip.com>
Also avoid prompting for full name time more than once.
Adds TOS version field to Remote server user.
Co-authored-by: Karl Stolley <karl@zulip.com>
Co-authored-by: Aman Agrawal <amanagr@zulip.com>
Adds three columns to the remote server activity chart and updates
the chart key for the third of those columns.
The first is the plan name. If there are multiple plans with a
status under the live threshhold, then we send "See support view".
The second is the plan status. If there are multiple plans, then
we send "Multiple plans".
The third is the estimated annual revenue for the plan. Note that
for free trials, this will be calculated as if the plan was paid
for 12 months (so a full year).
If there is no plan for the server under the live threshold or at
all then "---" is inserted into the table row. Note that 100%
sponsored servers/realms would fall into this category.
This is a general link for logging into the billing system on behalf of
a server, but it's tied to the .contact_email and takes the user
straight to the /deactivate/ page via the next_page mechanism.
This prep commit makes 'sync_license_ledger_if_needed'
function a 'BillingSession' abstract method.
We'll override the method for RemoteServerBillingSession
in the next commit.
This prep commit extracts out the code block that determines the
last license ledger for the customer plan having automanage_licenses
set to True into a new BillingSession method named
'get_last_ledger_for_automanaged_plan_if_exists'.
We'll be using this function while implementing the
'sync_license_ledger_if_needed' method for RemoteServerBillingSession.
We already calculate the correct `billed_licenses` early in the
function, so just used that to fix the bug where a legacy server
scheduled for upgrade doesn't respect the manual license count
set by the user.
Prakhar Pratyush
Lauryn Menard
Mateusz Mandera
Anders Kaseorg
Alya Abbott
Aman Agrawal
Tim Abbott
Alex Vandiver
Sahil Batra
Karl Stolley
Karl Stolley
Alya Abbott