Commit Graph

500 Commits

Author SHA1 Message Date
Steve Howell 31eb97ddde performance: Fix do_mark_stream_messages_as_read.
This function no longer asks for data that it
doesn't need.
2020-10-16 12:58:11 -07:00
Anders Kaseorg f0ae2a88f2 activity: Show server time rather than browser time.
This gets rid of an inline <script> that was blocking #16451.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-10-08 19:17:35 -07:00
Abhijeet Prasad Bodas bd61b570aa stats: Fix strings passed to _() not translated.
Part of #16094.
Strings constructed by _() were not being
translated in the /stats page.
This was because session variable was not set.
Ideally this should have been a part of b82bda9.
2020-10-02 14:56:20 -07:00
Vishnu KS 45b7673f4a support: Show realm string_id instead of name in alerts.
Since realm.name can be empty.
2020-09-28 15:37:49 -07:00
Vishnu KS 510efbc1a8 support: Add option to change billing method. 2020-09-28 15:37:49 -07:00
Vishnu KS 3e438538b4 support: Add support for downgrading realm. 2020-09-28 15:37:49 -07:00
Abhijeet Prasad Bodas b82bda9fb8 stats: Fix i18n not working in stats page.
Part of #16094.

Strings tagged with i18n were not being translated on the stats page.
This was because the translation data wasn't being sent to the front
end for this page.  That logic will be required in any page with a
bundle containing i18n JavaScript.
2020-09-25 16:22:34 -07:00
Anders Kaseorg 0228acf0f5 rest: Add rest_path shortcut for path with rest_dispatch.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 10:51:00 -07:00
Anders Kaseorg 5297e4a30a urls: Use unqualified imports.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 10:51:00 -07:00
Anders Kaseorg e70f2ae58d rest: Specify rest_dispatch handlers by function, not by string.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 10:46:28 -07:00
Anders Kaseorg faf600e9f5 urls: Remove unused URL names and shorten others.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-22 10:46:28 -07:00
Alex Vandiver 536bd3188e middleware: Move locale-setting before domain checking.
Calling `render()` in a middleware before LocaleMiddleware has run
will pick up the most-recently-set locale.  This may be from the
_previous_ request, since the current language is thread-local.  This
results in the "Organization does not exist" page occasionally being
in not-English, depending on the preferences of the request which that
thread just finished serving.

Move HostDomainMiddleware below LocaleMiddleware; none of the earlier
middlewares call `render()`, so are safe.  This will also allow the
"Organization does not exist" page to be localized based on the user's
browser preferences.

Unfortunately, it also means that the default LocaleMiddleware catches
the 404 from the HostDomainMiddlware and helpfully tries to check if
the failure is because the URL lacks a language component (e.g.
`/en/`) by turning it into a 304 to that new URL.  We must subclass
the default LocaleMiddleware to remove this unwanted functionality.

Doing so exposes a two places in tests that relied (directly or
indirectly) upon the redirection: '/confirmation_key'
was redirected to '/en/confirmation_key', since the non-i18n version
did not exist; and requests to `/stats/realm/not_existing_realm/`
incorrectly were expecting a 302, not a 404.

This regression likely came in during f00ff1ef62, since prior to
that, the HostDomainMiddleware ran _after_ the rest of the request had
completed.
2020-09-14 22:16:09 -07:00
Anders Kaseorg 1c80188648 urls: Elide str converter, which is the default.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-12 11:21:40 -07:00
Anders Kaseorg a50fae89e2 python: Elide type=str from argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 16:17:14 -07:00
Anders Kaseorg b4597a8ca8 python: Elide default for store_{true,false} argparse arguments.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-03 16:17:14 -07:00
Anders Kaseorg b84260014a populate_analytics_db: Replace intermediate list with generator.
Followup to commit ab120a03bc (#16265).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 19:34:48 -07:00
Anders Kaseorg ab120a03bc python: Replace unnecessary intermediate lists with generators.
Mostly suggested by the flake8-comprehension plugin.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:15:41 -07:00
Anders Kaseorg 1ded51aa9d python: Replace list literal concatenation with * unpacking.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-09-02 11:15:41 -07:00
Tim Abbott c05315ace2 update_analytics_counts: Fix warning output. 2020-08-28 14:24:48 -07:00
Anders Kaseorg 51f993e084 python: Remove unittest.mock.Mock uses from production code.
It’s somewhat expensive to import and confuses mypy.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-28 11:34:09 -07:00
Anders Kaseorg 61d0417e75 python: Replace ujson with orjson.
Fixes #6507.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-11 10:55:12 -07:00
Vishnu KS 4dc83a139c counts: Create 7day_actives::day counstat. 2020-08-10 17:22:19 -07:00
Anders Kaseorg 02fcb75ded analytics: Convert datetime to UNIX timestamp before JSON serialization.
datetime objects are not ordinarily JSON serializable.  While both
ujson and orjson have special cases to serialize datetime objects,
they do it in different ways.  So we want to do this explicitly.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-08-07 15:05:01 -07:00
arpit551 a68d38cc52 migrations: Upgrade migrations to remove duplicates in all Count tables.
This commit upgrades 0015_clear_duplicate_counts migration to remove
duplicate count in StreamCount, UserCount, InstallationCount as well.

Fixes https://github.com/zulip/docker-zulip/issues/266
2020-07-30 15:18:00 -07:00
Vishnu KS 18ad35013f support: Return json error if the POST request is invalid. 2020-07-28 11:03:06 -07:00
Vishnu KS 5b0b1efb15 support: Add functionality to approve sponsorship requests.
This should make it much easier to process these requests.
2020-07-24 17:55:38 -07:00
Vishnu KS 1a1396d07e support: Show customer plan details in support page. 2020-07-24 17:37:41 -07:00
Tim Abbott bcab06509a analytics: Remove unused analytics management commands.
Unlike stream_stats, I'm not aware of any of these having been used in
the last few years, and it's basically just really bad subsets of the
data in /activity, which also doesn't require shell access to use.

These haven't had real work or usage, AFAIK, since 2013.
2020-07-24 13:10:43 -07:00
Vishnu KS 49d06b9d69 activity: Always include realms on standard plan. 2020-07-24 13:09:43 -07:00
Steve Howell c44500175d database: Remove short_name from UserProfile.
A few major themes here:

    - We remove short_name from UserProfile
      and add the appropriate migration.

    - We remove short_name from various
      cache-related lists of fields.

    - We allow import tools to continue to
      write short_name to their export files,
      and then we simply ignore the field
      at import time.

    - We change functions like do_create_user,
      create_user_profile, etc.

    - We keep short_name in the /json/bots
      API.  (It actually gets turned into
      an email.)

    - We don't modify our LDAP code much
      here.
2020-07-17 11:15:15 -07:00
arpit551 87aaa84b42 audit_log: Log acting_user in do_change_user_role. 2020-07-06 17:32:11 -07:00
arpit551 19a8841a9e audit_log: Log acting_user in do_scrub_realm. 2020-07-06 17:24:18 -07:00
Vishnu KS 1e68525f83 support: Ensure that only one form is posted at a time.
The forms to change plan_type, add discount, scrub_realm etc
all post to the same endpoint.

Our frontend code is written so that only one form posts at a time.
But there should be no harm in enforcing the same in backend as well.
2020-07-01 16:45:38 -07:00
Vishnu KS 4c6350fa4b billing: Add option to request a sponsorship in /upgrade. 2020-07-01 16:45:38 -07:00
sahil839 1f8f227444 models: Update values of PreregistrationUser.invite_as dict.
This commit changes the PreregistrationUser.invite_as dict to have
same set of values as we have for UserProfile.role.

This also adds a data migration to update the already exisiting
PreregistrationUser and MultiuseInvite objects.
2020-06-24 11:09:07 -07:00
wowol 035d047dff urls: Migrate analytics urls to use modern django patterns. 2020-06-23 15:02:42 -07:00
Anders Kaseorg 3916ea23a9 python: Combine some split import groups.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-18 15:54:11 -07:00
Anders Kaseorg f364d06fb5 python: Convert percent formatting to .format for translated strings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-15 16:24:46 -07:00
Anders Kaseorg 5dc9b55c43 python: Manually convert more percent-formatting to f-strings.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
Anders Kaseorg 74c17bf94a python: Convert more percent formatting to Python 3.6 f-strings.
Generated by pyupgrade --py36-plus.

Now including %d, %i, %u, and multi-line strings.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
Anders Kaseorg 4b6d2cf25f logging: Pass more format arguments to logging.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
Anders Kaseorg 1a3441dbf5 confirmation: Pass realm rather than host to confirmation_url.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-14 23:27:22 -07:00
arpit551 aa70baba71 analytics: Add backend for messages read over time graph.
This commit includes changes in the api /json/analytics/chart_data to
send data for the newly added graph, as well as tests.
2020-06-14 21:19:24 -07:00
arpit551 a4b857b635 analytics: Populate data for messages_read::hour in development.
Generate data for messages_read::hour so that we can test messages_
read_over_time graph.
2020-06-14 21:19:14 -07:00
arpit551 c4b5d09283 analytics: Add LoggingCount for messages read stats.
Whenever we use API queries to mark messages as read we now increment
two new LoggingCount stats, messages_read::hour and
messages_read_interactions::hour.

We add an early return in do_increment_logging_stat function if there
are no changes (increment is 0), as an optimization to avoid
unnecessary database queries.

We also log messages_read_interactions::hour Logging stat
as the number of API queries to mark messages as read.

We don't include tests for the case where do_update_pointer is called
because do_update_pointer will most likely be removed from the
codebase in the near future.
2020-06-14 21:15:27 -07:00
arpit551 27daf38587 analytics: Derive AnalyticsTestCase class from ZulipTestCase.
The ZulipTestCase class contains various useful utility functions that
we could be using in test_counts.
2020-06-14 21:08:24 -07:00
Anders Kaseorg 0d6c771baf python: Guard against default value mutation with read-only types.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:31:27 -07:00
Anders Kaseorg 69c0959f34 python: Fix misuse of Optional types for optional parameters.
There seems to have been a confusion between two different uses of the
word “optional”:

• An optional parameter may be omitted and replaced with a default
  value.
• An Optional type has None as a possible value.

Sometimes an optional parameter has a default value of None, or None
is otherwise a meaningful value to provide, in which case it makes
sense for the optional parameter to have an Optional type.  But in
other cases, optional parameters should not have Optional type.  Fix
them.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-13 15:31:27 -07:00
Anders Kaseorg 365fe0b3d5 python: Sort imports with isort.
Fixes #2665.

Regenerated by tabbott with `lint --fix` after a rebase and change in
parameters.

Note from tabbott: In a few cases, this converts technical debt in the
form of unsorted imports into different technical debt in the form of
our largest files having very long, ugly import sequences at the
start.  I expect this change will increase pressure for us to split
those files, which isn't a bad thing.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-06-11 16:45:32 -07:00
Anders Kaseorg 69730a78cc python: Use trailing commas consistently.
Automatically generated by the following script, based on the output
of lint with flake8-comma:

import re
import sys

last_filename = None
last_row = None
lines = []

for msg in sys.stdin:
    m = re.match(
        r"\x1b\[35mflake8    \|\x1b\[0m \x1b\[1;31m(.+):(\d+):(\d+): (\w+)", msg
    )
    if m:
        filename, row_str, col_str, err = m.groups()
        row, col = int(row_str), int(col_str)

        if filename == last_filename:
            assert last_row != row
        else:
            if last_filename is not None:
                with open(last_filename, "w") as f:
                    f.writelines(lines)

            with open(filename) as f:
                lines = f.readlines()
            last_filename = filename
        last_row = row

        line = lines[row - 1]
        if err in ["C812", "C815"]:
            lines[row - 1] = line[: col - 1] + "," + line[col - 1 :]
        elif err in ["C819"]:
            assert line[col - 2] == ","
            lines[row - 1] = line[: col - 2] + line[col - 1 :].lstrip(" ")

if last_filename is not None:
    with open(last_filename, "w") as f:
        f.writelines(lines)

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-06-11 16:04:12 -07:00