Commit Graph

38269 Commits

Author SHA1 Message Date
Tim Abbott 29c66cf7c2 actions: Remove log_event and its legacy settings.
Now that we've finally converted these to use RealmAuditLog, we can
remove this ultra-legacy bit of code.
2020-07-24 12:13:16 -07:00
arpit551 0d79b55b2e audit_log: Record RealmAuditLog in do_change_notification_settings.
Removed logging with log_event and used RealmAuditLog instead.
Added tests in test_audit_log for the same.
2020-07-24 12:04:40 -07:00
arpit551 54df9290b9 audit_log: Record RealmAuditLog in do_rename_stream.
Removed logging with  log_event and used RealmAuditLog instead.
Added tests in test_audit_log for the same.
2020-07-24 12:00:31 -07:00
arpit551 94d2de8b4a audit_log: Record RealmAuditLog while changing default streams.
Removed logging with log_event and used RealmAuditLog instead.
Added tests in test_audit_log for the same.
2020-07-24 12:00:31 -07:00
arpit551 753713fde7 audit_log: Record RealmAuditLog in do_change_subscription_property.
Removed logging with log_subscription_property_change
and used RealmAuditLog instead.
Added tests in test_audit_log for the same.
2020-07-24 12:00:29 -07:00
arpit551 e6f54a3278 audit_log: Record RealmAuditLog in do_change_icon_source.
Removed logging with  log_event and used RealmAuditLog instead.
Added tests in test_audit_log for the same.
2020-07-24 11:56:40 -07:00
arpit551 12a19b0ff1 actions.py: Removed unnecessary logging in notify_subscription_added/removed.
We already record RealmAuditLog in bulk_add/remove_subscription
so there is no need to log while notifying.
2020-07-24 11:51:47 -07:00
arpit551 ccdf52fef6 audit_log: Log RealmAuditLog for realm notification properties.
Log RealmAuditLog for do_set_realm_notifications_stream and
do_set_realm_signup_notifications_stream function.
Added tests for the same.
2020-07-24 11:51:47 -07:00
arpit551 8f6a1c3f40 actions.py: Removed do_set_realm_message_deleting function.
Used do_set_realm_property function instead of
do_set_realm_message_deleting and removed it.
2020-07-24 11:51:47 -07:00
arpit551 422fa0ff7d audit_log: Log RealmAuditLog for do_set_realm_message_editing.
Log RealmAuditLog for do_set_realm_message_editing. Added tests for
same.
2020-07-24 11:51:46 -07:00
Anders Kaseorg ea7effbe8a recent_topics: Skip non-stream messages in update_topics_of_message_ids.
Fixes an exception when deleting a private message, introduced by
commit 5d5434ec2f (#15803).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-24 10:58:21 -07:00
Vinit Singh 96f579287a i18n: Tag markdown_help text for translation. 2020-07-24 10:54:34 -07:00
Vinit Singh fdbab54614 i18n: Translate Edit/View Source button's hmtl tooltip text.
This commit tags the html tooltip text for internationalization.
2020-07-24 10:54:34 -07:00
Vishnu KS 6cbbdcf37d mailmap: Fix the mailmap entry for Rishi Gupta.
Rishi never had a zulip.com email id. Giving preference to
zulip.com email id resulted in GitHub marking Rishi's commit
as anonymous. So we give preference to zulipchat.com instead,
which is linked to his GitHub account.
2020-07-24 10:51:47 -07:00
Vishnu KS 11a410e792 mailmap: Add mailmap entry for Reid Barton.
This is the email mentioned in commits that
are linked to Reid's GitHub account.
2020-07-24 10:51:47 -07:00
Vishnu KS 76b892fbc2 mailmap: Add mailmap entry for Scott Feeney. 2020-07-24 10:51:47 -07:00
Vishnu KS cc3e7ac90a mailmap: Add mailmap entry for Kevin Mehall.
Both the emails don't have an associated GitHub account,
but the entry will combine the the two accounts into one
in team page.
2020-07-24 10:51:47 -07:00
Vishnu KS cb293f135e mailmap: Add mailmap entry for Allen Rabinovich. 2020-07-24 10:51:47 -07:00
Vishnu KS b142a7ec83 mailmap: Add mailmap entry for Jeff Arnold. 2020-07-24 10:51:47 -07:00
Vishnu KS 9a8964c593 mailmap: Add mailmap entry for Jesscia McKellar.
Give preference to the email mentioned in commits that
are associated to Jessica's GitHub account.
2020-07-24 10:51:47 -07:00
Vishnu KS 3ec64b6092 team: Include users without an associated GitHub profile.
Including anon=1 in API requests will retrieve all contributors
of the repo. If there is no asscoiated GitHub account present for
the commits then the email and name of the author mentioned in
commit messages is returned.
2020-07-24 10:51:47 -07:00
Vishnu KS ecd0530578 tools: Rename server to zulip in fetch-contributor-data.
This was missed out in 2967294398
2020-07-24 10:51:47 -07:00
Steve Howell ce6e250e9d tests: Avoid "import logging" in test_transfer. 2020-07-24 10:45:59 -07:00
Steve Howell 996324aa0f tests: Avoid "import logging" for importer tests.
We now just use assertLogs.
2020-07-24 10:45:59 -07:00
Priyank Patel 98d1c215de github-actions: Rename focal and bionic tests job name to unit_tests. 2020-07-24 10:39:19 -07:00
Priyank Patel 07b79833a9 github-actions: Only run production suite on production related updates.
The production suite will only run if anything under puppet,
scripts, and tools or any migrations are updated. The '**' glob
means it includes subdirectory updates. For migrations all ~5
migrations directories are includes using the **/migrations/**
pattern.

The GitHub Action docs that explain the syntax:
https://docs.github.com/en/actions/reference/workflow-syntax-for-github-actions#filter-pattern-cheat-sheet
2020-07-24 10:36:55 -07:00
Priyank Patel 3ab53c6a83 github-actions: Add bionic and focal production install jobs.
We remove the "Do memcached hack" step from CircleCI when migrating
this job because a fix has been made upstream.
2020-07-24 10:36:55 -07:00
Priyank Patel 23372a49e1 github-actions: Add bionic production build job.
All the steps are same from circleci except two steps:
  1. The 'Add permissions ...' step is Actions specific as explained
     in comments.
  2. The step that used upload-artifacts is Actions verison of
     presist_to_workspace.

Finally, I should note the duplication in this and zulip-ci
workflow. There are three reason this is not a problem:
  1. It will be messy to mush this into zulip-ci workflow only for
     benefit of un-duplicating the env and cache restore steps.
  2. We needs this on its own workflow if we want to only run it
     when production related dependencies are updated.
  3. I don't see us updating the duplicated steps between both
     workflow. Circle CI config is prefect example for this; nothing
     is changed except for adding or updating steps which are not
     duplicated.
2020-07-24 10:36:55 -07:00
Priyank Patel 4c475ab2f7 github-actions: Do not fail other jobs when one job in matrix fails.
This change makes it so if focal backend job fails the bionic
backend and frontend jobs keeps running. Previously, it failed both
of the jobs if one failed. This is expected since typically matrix
is used to run sames tests on multiple versions and such but our use
case is bit more than that.
2020-07-24 10:36:55 -07:00
Priyank Patel 6a4792e844 tools/ci: Adjust ZULIP_PATH so it works for both CIs.
Until, we remove the circleci.
2020-07-24 10:36:55 -07:00
Priyank Patel 75ab26771b ci: Store production built assets to /tmp/production-build.
Previously, we copied them to /tmp and from there we specified those
assets we copied in circleci config in presist_to_workspace step.
Copying it to a directory allows us to get rid of list in circleci
config and GitHub Actions's upload artifact (their version of
presist to workspace) doesn't allow us to specify indivivual files
so only is this cleaner but required.
2020-07-24 10:36:55 -07:00
orientor c082fe301d openapi: Extract Emoji Reaction schema from `Messages`.
The Emoji Reaction schema is used in multiple place and hence
it is better to have it extracted as another schema in `components`.
2020-07-24 09:54:03 -07:00
orientor 881d57806b openapi: Move `stream` schema to components section.
The stream schema is used in two locations so move it to the
components section. Also the `is_default` key returned by `/streams`
is not returned by `/events`. So handle it separately.
2020-07-24 09:51:31 -07:00
orientor 2b879ecbdf openapi: Edit `Messages` component to match the `message` event object.
The `Messages` schema present in `#/components/schemas` was a
combination of all keys possible in any message object used in Zulip.
Edit it so that the original `Messages` contains just the keys present
in the `message` event. Also make another schema  `GetMessages` which
adds a few other keys which are received when using `GET /messages`.
The message object in the `/zulip-outgoing-webhook` has also been
modified and corrected.
2020-07-24 09:51:31 -07:00
orientor bdf3ecea42 openapi: Move `subscriptions` schema to the components section.
The `subscriptions` has use in multiple endpoints and hence instead
of redefining it at every point move it to the the components section
for easier reuse.
2020-07-24 09:51:31 -07:00
Anders Kaseorg e3b3df328d eslint: Replace sort-imports with import/order.
import/order sorts require() calls as well as import statements.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-24 09:42:56 -07:00
Anders Kaseorg 0771e7fc59 eslint: Add import plugin.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-24 09:42:56 -07:00
Anders Kaseorg a83c2c6296 eslint: Make subdirectory configurations more consistent.
Enable ES2020 everywhere except Casper.  Disable Node-specific globals
in static.  Disable browser-specific and CommonJS globals in shared.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-24 09:42:56 -07:00
Anders Kaseorg c9ce77c28b eslint: Use babel-eslint parser.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-24 09:42:56 -07:00
Anders Kaseorg 2e029eb58b lint: Use Prettier for JSON files.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-24 09:42:56 -07:00
Anders Kaseorg 40be4df57a lint: Format JSON files with Prettier.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-24 09:42:56 -07:00
Steve Howell f03605bd73 event_schema: Support plan_type in check_realm_update. 2020-07-24 09:38:34 -07:00
Steve Howell 33f173ae1b event_schema: Use check_realm_update in two more places.
We also have the caller pass in the property name for an
additional sanity check.

Note that we don't yet handle the possibility of extra_data;
that will be a subsequent commit.

Also, the stream_id fields aren't in Realm.property_types,
so we specify their types in the checker.
2020-07-24 09:38:34 -07:00
Steve Howell 176ab66fc7 event_schema: Extract check_realm_user_update.
This a pretty big commit, but I really wanted it
to be atomic.

All realm_user/update events look the same from
the top:

    _check_realm_user_update = check_events_dict(
        required_keys=[
            ("type", equals("realm_user")),
            ("op", equals("update")),
            ("person", _check_realm_user_person),
        ]
    )

And then we have a bunch of fields for person that
are optional, and we usually only send user_id plus
one other field, with the exception of avatar-related
events:

    _check_realm_user_person = check_dict_only(
        required_keys=[
            # vertical formatting
            ("user_id", check_int),
        ],
        optional_keys=[
            ("avatar_source", check_string),
            ("avatar_url", check_none_or(check_string)),
            ("avatar_url_medium", check_none_or(check_string)),
            ("avatar_version", check_int),
            ("bot_owner_id", check_int),
            ("custom_profile_field", _check_custom_profile_field),
            ("delivery_email", check_string),
            ("full_name", check_string),
            ("role", check_int_in(UserProfile.ROLE_TYPES)),
            ("email", check_string),
            ("user_id", check_int),
            ("timezone", check_string),
        ],
    )

I would start the code review by just skimming the changes
to event_schema.py, to get the big picture of the complexity
here.  Basically the schema is just the combined superset of
all the individual schemas that we remove from test_events.

Then I would read test_events.py.

The simplest diffs are basically of this form:

    -  schema_checker = check_events_dict([
    -      ('type', equals('realm_user')),
    -      ('op', equals('update')),
    -      ('person', check_dict_only([
    -          ('role', check_int_in(UserProfile.ROLE_TYPES)),
    -          ('user_id', check_int),
    -      ])),
    -  ])

    # ...
    -  schema_checker('events[0]', events[0])
    +  check_realm_user_update('events[0]', events[0], {'role'})

Instead of a custom schema checker, we use the "superset"
schema checker, but then we pass in the set of fields that we
expect to be there.  Note that 'user_id' is always there.

So most of the heavy lifting happens in this new function
in event_schema.py:

    def check_realm_user_update(
        var_name: str, event: Dict[str, Any], optional_fields: Set[str],
    ) -> None:
        _check_realm_user_update(var_name, event)

        keys = set(event["person"].keys()) - {"user_id"}
        assert optional_fields == keys

But we still do some more custom checks in test_events.py.

custom profile fields: check keys of custom_profile_field

     def test_custom_profile_field_data_events(self) -> None:
+        self.assertEqual(
+            events[0]['person']['custom_profile_field'].keys(),
+            {"id", "value", "rendered_value"}
+        )

+        check_realm_user_update('events[0]', events[0], {"custom_profile_field"})
+        self.assertEqual(
+            events[0]['person']['custom_profile_field'].keys(),
+            {"id", "value"}
+        )

avatar fields: check more specific types, since the superset
    schema has check_none_or(check_string)

     def test_change_avatar_fields(self) -> None:
+        check_realm_user_update('events[0]', events[0], avatar_fields)
+        assert isinstance(events[0]['person']['avatar_url'], str)
+        assert isinstance(events[0]['person']['avatar_url_medium'], str)

+        check_realm_user_update('events[0]', events[0], avatar_fields)
+        self.assertEqual(events[0]['person']['avatar_url'], None)
+        self.assertEqual(events[0]['person']['avatar_url_medium'], None)

Also note that avatar_fields is a set of four fields that
are set in event_schema.

full name: no extra work!

     def test_change_full_name(self) -> None:
-        schema_checker('events[0]', events[0])
+        check_realm_user_update('events[0]', events[0], {'full_name'})

test_change_user_delivery_email_email_address_visibilty_admins:

    no extra work for delivery_email
    check avatar fields more directly

roles (several examples) -- actually check the specific role

     def test_change_realm_authentication_methods(self) -> None:
-            schema_checker('events[0]', events[0])
+            check_realm_user_update('events[0]', events[0], {'role'})
+            self.assertEqual(events[0]['person']['role'], role)

bot_owner_id: no extra work!

-        change_bot_owner_checker_user('events[1]', events[1])
+        check_realm_user_update('events[1]', events[1], {"bot_owner_id"})

-        change_bot_owner_checker_user('events[1]', events[1])
+        check_realm_user_update('events[1]', events[1], {"bot_owner_id"})

-        change_bot_owner_checker_user('events[1]', events[1])
+        check_realm_user_update('events[1]', events[1], {"bot_owner_id"})

timezone: no extra work!

-                timezone_schema_checker('events[1]', events[1])
+                check_realm_user_update('events[1]', events[1], {"email", "timezone"})
2020-07-24 09:38:34 -07:00
Steve Howell 38bd66d8ae test flake fix: Avoid logging leak for webhook tests.
We can still improve these tests to use assertLogs
context managers, but this stops the tests from
having logging side effects via setUp.
2020-07-24 10:56:42 -04:00
Tim Abbott b840a8b3ed i18n: Update translation data from Transifex. 2020-07-23 12:06:15 -07:00
Gittenburg f9459bba12 upload: Do not open compose box when editing.
Previously editing a message and uploading a file in
the edit textarea opened the message compose box.

Fixes #15890.
2020-07-23 11:29:32 -07:00
Anders Kaseorg e123f8f723 subs: Fix set_muted parameter order.
The status_element parameter is optional, and the other caller in
stream_popover.js does not provide it.  This fixes a regression in
commit e6a66063a9 (#15868).

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-23 11:22:47 -07:00
Tim Abbott 4a7eb47c36 test_push_notifications: Use assertLogs for bouncer errors. 2020-07-23 10:54:13 -07:00
Anders Kaseorg ca42fc2e21 casper_tests: Log in as Desdemona, not Iago.
As of commit 87e72ac8e2 (#15267), we
need to be an owner for some of the tested functionality, not just an
administrator.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-23 10:26:27 -07:00