Commit Graph

34877 Commits

Author SHA1 Message Date
majordwarf 7faaed680b css: Use SCSS nesting in stats.scss for `#id_messages_sent_over_time`. 2020-03-05 13:57:01 -08:00
majordwarf 930e8704c0 css: Use SCSS nesting in stats.scss for `.button`. 2020-03-05 13:57:01 -08:00
majordwarf f18f1e547c css: Use SCSS nesting in stats.scss for `.buttons`. 2020-03-05 13:57:01 -08:00
majordwarf 1ebb7c1c89 css: Use SCSS nesting in stats.scss for `.pie-chart`. 2020-03-05 13:57:01 -08:00
majordwarf 733cb116fc css: Use SCSS nesting in stats.scss for `.chart-container`. 2020-03-05 13:57:01 -08:00
majordwarf fdfcbfbaa6 css: Reorder stats.scss to put similar elements next to each other. 2020-03-05 13:57:01 -08:00
majordwarf da5651435e css: Remove redundant rules for `#users_hover_info` and `#hoverinfo`.
The copy of the styling for users_hover_info:

```
-#users_hover_info {
-    left: 25px;
-    top: -40px;
-}
```

Looks less good than the common one with #hoverinfo, so we remove it.
2020-03-05 13:57:01 -08:00
Divyanshu Agrawal c0e19ed905 settings page: Change cursor for date input to 'pointer'.
Currently, the cursor for the date input field in the settings page
is 'not-allowed' as it has the disabled attribute because we want
users to pick the date from the date picker. But this leads to
confusion whether the field is editable at all.

Change the cursor to 'pointer' to make it clear that the field has
a click action associated with it.
2020-03-05 11:10:42 -08:00
Tim Abbott 56591890b0
org settings: Fix bot owner profile display.
Clicking on the 'Owner' value for a row in the list of bots does
nothing, and causes a blueslip error.

This is because the map object in which we store the users have
integer keys, while we pass the owner id as string.

This is fixed by parsing the owner id to integer before passing it
on.

Fixes #14107.
2020-03-05 11:02:54 -08:00
orientor 7d2bb707e7 integrations: Improved Taiga Integration.
This adds links to the users and other small improvements.

Progress towards #13698.
2020-03-04 16:52:52 -08:00
Tim Abbott 93438d965d docs: Update a few GSoC project ideas.
There are more updates like this coming in the next few days.
2020-03-04 16:48:58 -08:00
Yashwant 849adc8fa4 docs: Fix last date for gsoc proposal submission. 2020-03-04 15:06:37 -08:00
Yashwant 02c7131a64 Fixed last date of proposal submission GSOC 2020-03-04 15:06:34 -08:00
majordwarf 5181ca9a66 css: Reorder stats.scss so `.pie-chart` are in same place. 2020-03-04 14:25:54 -08:00
majordwarf 4e8deb1e50 css: Reorder stats.scss so `.chart-container` are in same place. 2020-03-04 14:25:54 -08:00
Tim Abbott a2cf341d64 help: Add some emoji reactions -> emoji links. 2020-03-03 16:45:38 -08:00
Mateusz Mandera 2d544250b7 events: Add block for compatibility with old delete_message events. 2020-03-03 15:52:42 -08:00
Mateusz Mandera 3922fb3a92 events: Clean up delete_message even processing code. 2020-03-03 15:52:42 -08:00
Mateusz Mandera e506dbcdad auth: Monkey patch a fix for Github deprecation notice spam.
This is a way to monkey-patch a fix for
https://github.com/python-social-auth/social-core/issues/430
Changes from this commit should be reverted once the issue is fixed
upstream.
2020-03-03 15:51:40 -08:00
Mateusz Mandera 03a6da27f1 auth: Fix Github auth with organization/team membership restriction.
We need to request access to read:org scope to be able to check org/team
membership. Without it SOCIAL_AUTH_GITHUB_ORG_NAME and
SOCIAL_AUTH_GITHUB_TEAM_ID settings don't work and simply lead to all
auth attempts failing.
Tested manually.
2020-03-03 15:51:14 -08:00
Tim Abbott 783a77c532 queue processors: Flush per-request caches after each item.
Several of our queues are capable of doing work that includes
rendering markdown (outgoing_webhook, embedded_bots, embed_links, and
email_mirror).  As a result, it's essential that these don't cache
per-request data (specifically, realm filters) longer than they
should, making editing/deleting linkifiers potentially use old
settings until the relevant process was restarted.

Flushing these caches is extremely cheap (just clearing two
dictionaries) and thus is reasonable to do after every queue event,
rather than trying to do it only the ~1/3 of queues that specifically
do markdown processing.  We do the same in our middleware for
reset_queries.

It's not worth writing a test for this because it's very difficult to
create the test setup situation for this bug with a single test worker
process; one needs to edit the linkifier configuration in a different
process than the one sending the message in order to see the bug.

This was a much larger visible bug on Zulip 2.1.x, where the presence
of the message_sender queue meant that this would apply to messages
sent via a browser.

Fixes #14095.
2020-03-03 15:29:11 -08:00
Rohitt Vashishtha ff5e2b6eb7 bugdown: Avoid hanging list paragraphs being processed as codeblocks.
Previously, the input:

====================
- One
  - Two

    Two continued
====================

Would produce the same output as:

====================
- One
  - Two

```
Two continued
```
====================

This was because our CodeBlockProcessor had a higher priority than
the ListIndentProcessor. This issue was discussed here:
https://chat.zulip.org/#narrow/stream/9-issues/topic/continuation.20paragraphs.20in.20list.20items.
2020-03-03 12:08:19 -08:00
Rohitt Vashishtha cd7396e732 bugdown: Update outdated comment about Zulip's heading support. 2020-03-03 11:54:18 -08:00
Steve Howell 862515b7a4 presence: Avoid failures with obsolete events.
We only recently added `user_id` to presence
events.
2020-03-03 11:45:45 -08:00
Anders Kaseorg eee784312d stats: Fix Other calculation.
In commit 2248d4d9a4 (#13883) I
incorrectly migrated a loop.  Fixes #14052.

Signed-off-by: Anders Kaseorg <anders@zulipchat.com>
2020-03-02 17:45:44 -08:00
Rohitt Vashishtha 62a7e464fb bugdown: Use AtomicString in StreamPattern.
This fixes the stream counterpart of #14080.
2020-03-02 00:03:33 -08:00
Rohitt Vashishtha 245de9e1e2 bugdown: Use AtomicString in StreamTopicPattern.
Fixes #14080.
2020-03-02 00:03:33 -08:00
Mateusz Mandera 05e7214690 do_delete_messages: Handle empty set of messages passed as input.
/delete_topic endpoint could be used to request the deletion of a topic,
that would cause do_delete_messages to be called with an empty set in
these cases:
1. Requesting deletion of an empty stream.
2. Requesting deletion of a topic in a private stream with history not
   public to subscribers, if the requesting admin doesn't have access to
   any of the messages in that topic.
2020-03-02 00:01:35 -08:00
Steve Howell 94192395fb perf: Extract Stream.get_client_data.
This function slims down the data that we get
from the database in order to create the
streams part of our client payload.

We also fix a typo.

We also clearly distinguish between queries
and lists here.
2020-03-01 22:38:03 -08:00
Steve Howell 49b8218463 perf: Extract get_subscribed_stream_ids_for_user.
This new method prevents us from getting fat
objects from the database.

Instead, now we just get ids from the database
to build our subqueries.

Note that we could also technically eliminate
the `set(...)` wrappers in this code to have
Django make a subquery and save a round trip.
I am postponing that for another commit (since
it's still somewhat coupled to some other
complexity in `do_get_streams` that I am trying
to cut through, plus it's not the main point
of this commit.)

BEFORE:

    # old, still in use for other codepaths
    def get_stream_subscriptions_for_user(user_profile: UserProfile) -> QuerySet:
        # TODO: Change return type to QuerySet[Subscription]
        return Subscription.objects.filter(
            user_profile=user_profile,
            recipient__type=Recipient.STREAM,
        )

    user_subs = get_stream_subscriptions_for_user(user_profile).filter(
        active=True,
    ).select_related('recipient')
    recipient_check = Q(id__in=[sub.recipient.type_id for sub in user_subs])

AFTER:

    # newly added
    def get_subscribed_stream_ids_for_user(user_profile: UserProfile) -> QuerySet:
        return Subscription.objects.filter(
            user_profile_id=user_profile,
            recipient__type=Recipient.STREAM,
            active=True,
        ).values_list('recipient__type_id', flat=True)

    subscribed_stream_ids = get_subscribed_stream_ids_for_user(user_profile)
    recipient_check = Q(id__in=set(subscribed_stream_ids))
2020-03-01 22:38:03 -08:00
Steve Howell eb368c9c92 performance: Optimize max_message_id calculation.
We calculate `max_message_id` for the mobile client.

Our query now no longer joins to the Message table
and just grabs one value instead of fat objects.
2020-03-01 22:38:03 -08:00
Mateusz Mandera 6e4dcc714e auth: Fix bug in backend configuration checks in start_social_login.
We shouldn't check if *every* backend in the backends list is
configured, but only the one the user is attempting to log in with.
2020-03-01 22:30:38 -08:00
Steve Howell 4fba227898 backend tests: Test include_all_active for streams.
We were only checking error handling before, not
the happy path.  The structure of the code
made it so that we effectively tested most of the
logic for this use case (since all the other flags
are sort of just filters on top of this), but
obviously we want explicit coverage here.  Also,
we weren't testing the is-admin-but-not-api-super-user
error checking until this commit.
2020-03-01 07:49:38 -05:00
Steve Howell d1d0b415f1 i18n: Rename translations.js to i18n.js.
The file populates `windows.i18n`, so now
the file name matches our convention.

Note that the module really just initializes
`i18next` and then does this:

    window.i18n = i18next;

It doesn't really add any functionality to
third party library.
2020-02-29 12:19:51 -08:00
Steve Howell e67be55152 node tests: Actually test `translations.js`.
Before this test, we were validating the behavior
of `i18next`, but we weren't validating our light
layer that sits on top of `i18next`, which currently
resides in the slightly misnamed `translations.js`
file.

The translations module is now so small that I'll
just quote it verbatim here:

    import i18next from 'i18next';

    i18next.init({
        lng: 'lang',
        resources: {
            lang: {
                translation: page_params.translation_data,
            },
        },
        nsSeparator: false,
        keySeparator: false,
        interpolation: {
            prefix: "__",
            suffix: "__",
        },
        returnEmptyString: false,  // Empty string is not a valid translation.
    });

    window.i18n = i18next;

We now just do `zrequire('translations')` to initialize
the `i18next` library, which allows us to have simpler
test setup and to actually exercise the above call to
`i18next.init`.

This change now gives us 100% line coverage of `translations.js`,
which of course isn't that hard to acheive (see above).
2020-02-29 12:19:51 -08:00
Steve Howell 1abd00eac2 i18n: Remove code to sweep local storage.
Before 2018, we used a feature of i18next where
we would cache translations in local storage
for up to two weeks:

    var cacheOptions = {
        // ...
        prefix: 'i18next:' + page_params.server_generation + ':',
        expirationTime: 2*7*24*60*60*1000,  // 2 weeks
    };

    i18next.init({
        /// ...
        cache: cacheOptions
    }

Because `server_generation` would change each time you
upgraded a server, a frequently upgraded server like
chat.zulip.org would cause its active users to start
to accumulate lots of obsolete key/value pairs in local
storage over the two weeks.

See #4443 for more details.

We eventually reduced the cache life to 2 days.  And then
on top of that, newer versions of the server would start
to clean up after themselves using this commit from
April 2017:

    e3f1d025ae

We then removed the caching option altogether a year
later in May 2018:

    cff40c557b

We kept around the code to remove all the old keys, though.
This was particularly important for users who may have
been hitting servers that did an upgrade to the new
version from some older version that didn't have the
key-fixing code.

But mostly the problem takes care of itself after
either two days or two weeks, even on really out-of-date
servers.

The original problem was most likely to affect server
admins that did a lot of upgrades (and possibly only really
affected chat.zulip.org), so as long as those server
admins continued their patterns, it's highly likely that
they've done several upgrades since May 2018 that would
have cleaned these keys out for good.

And, again, even if there is some strange straggler here,
they probably only have one set of keys that will expire
either two days or two weeks after an upgrade, depending
on how long ago the prior upgrade was.  (All of their
keys based on older versions of `server_generation` would
have long since expired.)

Finally, any upgrade certainly won't make the problem
worse for any users under this hypothetical situation,
since the new server won't be writing new keys.

So I am removing the cleanup code.
2020-02-29 12:19:51 -08:00
Tim Abbott 6c74af4c06 docs: Document our stable release branches.
The experiment with the published 2.1.x tpye branches has been going
well, so we should document the feature.
2020-02-29 11:50:44 -08:00
Steve Howell 979dcfe85b refactor: Extract settings_data.py.
This extracts a new module with three
functions, which we will test with 100%
line coverage:

    - show_email
    - email_for_user_settings
    - get_time_preferences

The first two break several dependencies
in the codebase on `settings_org.js`.  The
`get_time_preferences` breaks an annoying
dependency on `page_params` within people.

The module is pretty cohesive, in terms that
all three functions are just light wrappers
around `page_params` and/or `settings_config`.

Now all the modules that want to call show_email()
only have to require `settings_data`, instead of
having a dependency on the much heavier
`settings_org.js` module.

I also make some of the unit tests here be more
full-stack, where instead of stubbing show_email,
I basically just toggle `page_params.is_admin`.
2020-02-28 17:11:24 -08:00
Steve Howell b994889315 node tests: Just set i18n every time.
Explicitly stubbing i18n in 48 different files
is mostly busy work at this point, and it doesn't
provide much signal, since often it's invoked
only to satisfy transitive dependencies.
2020-02-28 17:11:24 -08:00
Steve Howell 588fa9d150 node tests: Make i18next more explicit in i18n tests.
We are actually testing i18next somewhat directly in
this test, so we clean up some names in the file.

We'll address filenames in another commit.
2020-02-28 17:11:24 -08:00
Mateusz Mandera 2d55c7e0d5 home: Don't assume user agent header is set for insecure_desktop_app.
The header may not be set - this leads to CI failures on 2.1.x branch,
but in any case is a real bug.
2020-02-28 16:51:23 -08:00
Chris Bobbe 23ba2b63c5 push_notifications: In dev, make APNs or GCM config suffice. 2020-02-28 16:49:35 -08:00
Steve Howell 504ec9d489 typing: Remove recipient-related complexity.
For historical reasons we were creating Recipient
objects at some point in the typing-notifications
codepath.  Now we just work with UserProfiles.
This removes some queries, as indicated by
the change to `len(queries)` in a couple of the
tests.

The one subtle thing that changes here is huddles.
If user 10 sends a typing notification that they
are talking to users 20 and 30, there might not
actually be a huddle for users 10/20/30, but
we were actually creating huddles on the fly!
There is no need to create huddles just for
typing notifications, since we don't even
share huddle ids with our clients.  The clients
just infer the huddles.

Some of the code that gets killed off here as
somewhat "collateral damage" is some
defensive code related to formerly supporting streams
in typing indicators.  The support for streams
was killed off almost as soon as we released
the feature, and the codepath is pretty clearly
user-centric at this point.
2020-02-28 12:46:20 -08:00
Steve Howell f224f215c1 refactor: Simplify handling of emails for typing endpoint.
Instead of duplicating code for the email case, just
convert emails to user_ids and then run the same code.
2020-02-28 12:39:36 -08:00
Steve Howell bed6d5a789 typing: Inline check_typing_notification.
I actually like this pattern:

    def check_send_typing_notification(...):
        typing_notification = check_typing_notification(...)
        do_send_typing_notification(...)

It can help divide responsibilities nicely and make it easy
to write detailed unit tests against each of the two helpers.

Unfortunately, the good things didn't really happen here, and
instead we got the worst aspects of the pattern:

    - The responsibilities for validation leaked into
      the second function.

    - Both functions were doing sane things individually
      that became not-so-sane in the big picture (namely,
      we ended up making Recipient objects for no reason,
      but if you read each of the helpers, it was just one
      step that seemed reasonable).

    - Passing around dictionaries for results can be annoying.

Also, the pattern made a lot more sense when the validation
for typing was a lot more complicated.  My prior commit makes
it so that we only ever deal with a list of user_ids.

Anyway, now I'm inlining it. :)

Subsequent commits will clean up the more substantive issue
here, which is that we are building Recipients for no reason.
2020-02-28 12:39:36 -08:00
Steve Howell b26f2dcd4b typing: Deprecate emails in typing endpoint.
The only clients that should use the typing
indicators endpoint are our internal clients,
and they should send a JSON-formatted list
of user_ids.

Unfortunately, we still have some older versions
of mobile that still send emails.

In this commit we fix non-user-facing things
like docs and tests to promote the user_ids
interface that has existed since about version
2.0 of the server.

One annoyance is that we documented the
typing endpoint with emails, instead of the
more modern user_ids, which may have delayed
mobile converting to user_ids (and which
certainly caused confusion).  It's trivial
to update the docs, but we need to short
circuit one assertion in the openapi tests.

We also clean up the test structure for the
typing tests:

    TypingHappyPathTest.test_start_to_another_user
    TypingHappyPathTest.test_start_to_multiple_recipients
    TypingHappyPathTest.test_start_to_self
    TypingHappyPathTest.test_start_to_single_recipient
    TypingHappyPathTest.test_stop_to_another_user
    TypingHappyPathTest.test_stop_to_self

    TypingValidateOperatorTest.test_invalid_parameter
    TypingValidateOperatorTest.test_missing_parameter

    TypingValidateUsersTest.test_argument_to_is_not_valid_json
    TypingValidateUsersTest.test_bogus_user_id
    TypingValidateUsersTest.test_empty_array
    TypingValidateUsersTest.test_missing_recipient

    TypingValidationHelpersTest.test_recipient_for_user_ids
    TypingValidationHelpersTest.test_recipient_for_user_ids_non_existent_id

    TypingLegacyMobileSupportTest.test_legacy_email_interface
2020-02-28 12:39:36 -08:00
Akash Nimare 97947bc381 desktop: Update desktop app to v4.0.3. 2020-02-28 12:04:18 -08:00
Tim Abbott d79a7a8c35 panels: Show a banner for users with legacy desktop apps.
Users who are using ZulipDesktop or haven't managed to auto-update to
ZulipElectron should be strongly encouraged to upgrade.

We'll likely want to move to something even stricter that blocks
loading the app at all, but this is a good start.
2020-02-28 01:54:46 -08:00
Mateusz Mandera 7db3d4560f do_delete_messages: Archive the messages in bulk.
The test added in this commit shows 37 queries - compared to 181 without
the change to the function. That seems very much worth it.
2020-02-27 23:12:32 -08:00
Mateusz Mandera b4186fb680 do_delete_messages: Remove unused message_ids list. 2020-02-27 23:12:32 -08:00