Commit Graph

38065 Commits

Author SHA1 Message Date
Mohit Gupta b0b3cf86ce tests: Move test_auto_subbed_to_personals test to test_signup.py.
This is really a test about account creation.
2020-07-08 13:59:56 -07:00
Mohit Gupta 478956ff0b tests: Move MiscMessageTest to test_audit_log.py. 2020-07-08 13:53:56 -07:00
Mohit Gupta f03c5e35e4 tests: Move NoRecipientIDsTest to test_subs.py. 2020-07-08 13:51:32 -07:00
Vinit Singh 8b54cce833 emoji_picker: Improve emoji picker search results.
This sorts the emoji picker's search results using the same
`typehead.sort_emojis` function as the compose typehead.

Resolves #15694.
2020-07-08 13:47:03 -07:00
Gittenburg 6e5fb85ade refactor: Improve method name in emoji_picker. 2020-07-08 13:25:49 -07:00
Gittenburg 9f20fdc87a refactor: Replace getter with const in emoji_picker. 2020-07-08 13:25:49 -07:00
Gittenburg 08b9aacb88 hotkey: Detect arrow keys while holding shift.
This is useful for the emoji-picker where holding down shift lets you
select multiple emojis. Otherwise when selecting multiple emojis with
the keyboard shortcuts you would need to release shift everytime you
wanted to navigate with the arrow keys.
2020-07-08 13:25:49 -07:00
Gittenburg 5bb10036b5 emoji_picker: Allow selecting multiple while holding Shift.
We detect shift with event.shiftKey so we need to pass the event around.

Fixes #8465.
2020-07-08 13:25:49 -07:00
Gittenburg 1113589b9d emoji_picker: Move enter handling into picker.
This means the picker no longer has to export
is_composition(emoji) and toggle_selected_emoji().
2020-07-08 13:22:24 -07:00
Gittenburg 42f20e81be emoji_picker: Fix inconsistency in edge case.
Previously clicking on an existing message reaction (outside of the
emoji picker) while having the emoji picker open, removed the reaction
without updating the highlighting of the reacted emoji in the emoji
picker.

The emoji picker already is already closed when clicking outside.
The message reaction click handler however previously stopped the event
propagation, leaving the picker open, allowing the inconsistency.
2020-07-08 13:21:53 -07:00
Tim Abbott 92d33070dd docs: Improve explanation of running master.
It's a bit annoying we haven't renamed master yet, but we'll do that
in a bulk pass.
2020-07-08 12:17:59 -07:00
Priyank Patel 61955ecf41 github-actions: Don't run code scanning workflow every week.
Since we already run this on every push we don't need to run it as a
cron job every week for no reason. While we are touching this code
block, we convert it to on: [push, pull_request] since the previous
format felt weird. It was only written that way because we had the
cron job declared there.
2020-07-07 17:26:16 -07:00
Priyank Patel 711943a3ab github-actions: Use include_frontend_test check.
Uses the include_frontend_test instead of platfrom specific check.
2020-07-07 17:26:16 -07:00
Priyank Patel 4938d6ab6b minor: Rename codeql workflow names.
This makes it so the GitHub displays the runs as "Code Scanning / CodeQL"
instead of "Code scanning - actions / CodeQL-Build".
2020-07-07 17:26:16 -07:00
Mohit Gupta 086e9e281b models: Remove to_log_dict method of Message model as it's unused. 2020-07-07 17:14:10 -07:00
Mohit Gupta de81b92dee tests: Move MessageVisibilityTest to test_message_fetch.py. 2020-07-07 17:14:10 -07:00
Mohit Gupta 012622aa4a tests: Move CheckMessageTest to test_message_send.py. 2020-07-07 17:14:10 -07:00
Mohit Gupta 4ae6cc7328 tests: Move MessageHasKeywordsTest to test_message_fetch.py. 2020-07-07 17:14:10 -07:00
Mohit Gupta ce948e4a80 tests: Move MessageAccessTests to test_message_flags.py. 2020-07-07 17:14:10 -07:00
Mohit Gupta 096ed90ac2 tests: Rename test_unread.py to test_message_flags.py. 2020-07-07 17:14:10 -07:00
Mohit Gupta aa7dc6bf83 tests: Move SewMessageAndReactionTest to test_message_dict.py. 2020-07-07 17:14:10 -07:00
Mohit Gupta 228fb9f80d tests: Move TestAddressee to test_message_send.py. 2020-07-07 17:14:10 -07:00
Mohit Gupta 329265b1bf tests: Move TestCrossRealmPMs to test_message_send.py. 2020-07-07 17:14:10 -07:00
Mohit Gupta 308643cace tests: Rename test_narrow.py to test_message_fetch.py.
This commit renames test_narrow.py and also all its references to
test_message_fetch.py.
2020-07-07 17:14:10 -07:00
sahil839 9ccb33a894 subs: Fix bug of subscribing/unsubscribing user by clicking on stream row.
There is a bug when clicking on the stream row in stream settings page
subscribes/unsubscribes the user from stream. The ideal behaviour
should be that user should be subscribed/unsubscribed only
when user clicks on tick icon. The buggy behaviour is visible only after
clicking the tick icon once.

This is because clicking on tick is adding sub_unsub_button class to the
stream row, while it should be added only to the ".check" element to
re-enable the button to handle requests.

This commit fixes the bug by adding the "sub_unsub_button" class to
".check" element only and not to the stream row and same change
is done for removing the "sub_unsub_button" class also.
2020-07-07 17:12:01 -07:00
sahil839 48eba187bb stream_edit: Fix bug in subscribing/unsubscribing from stream settings.
Subscribe/Unsubscribe button in the right section of stream settings
page is not working because the target element in click handler was
changed to 'check.sub_unsub_button' in c234b4f2 and the button in
the right section with class 'sub_unsub_button' also uses the same
click handler.

This commit reverts c234b4f2 and the bug of subscribing/unsubscribing
the user by clicking on stream row in the stream settings page, which
c234b4f2 intended to fix will be fixed in next commit.
2020-07-07 17:12:01 -07:00
Tim Abbott 8672805ddf compose: Use rendered_markdown.update_elements for previews.
As we add more features where rendered_markdown.update_elements does
something useful, it'll become important to run this code everywhere
we render markdown in the DOM.

One can see in this case that we had actually copied one hunk of
rendered_markdown.update_elements years ago, before we extracted it as
an independent function; we get to delete that copy.

Fixes #15500.
2020-07-07 14:37:09 -07:00
arpit551 4f6cd6577c http_headers: Substitute Nginx version based on OS.
success-http-headers-bionic.txt and success-http-headers-focal.txt
differ only in the nginx version so this substitution will allow
us to have single file for both of them. Also this change helps
to avoid CI failure if Nginx version is updated in the OS.
2020-07-07 11:20:05 -07:00
sahil839 04ef6ba63a compose: Set wildcard_mention for message before further validation.
We do not show the warning while sending messages to announce
stream if there is a wildcard mention in the message (i.e.
when wildcard_mention != null)

There are two cases where we should ideally show the warning
but we don't-

 - When there is no wildcard mention in the message and
   wildcard_mention is set to undefined (initial value of
   wildcard_mention).
   This is because "wildcard_mention != null" returns true for
   this case and thus the warning is not shown, assuming the
   message to have wildcard mention.

 - When previous message had a wildcard mention and now a message
   is being sent with no wildcard mention.
   This is because the condition "wildcard_mention != null" is
   checked with the previous value of wildcard_mention and not
   with the value according to current message content, and thus
   the warning is not shown, assuming the message to have wildcard
   mention.

This commit changes the code to set wildcard_mention from the
latest message content before performing other validations and
thus solves the problems described above.
2020-07-07 10:42:55 -07:00
Rohitt Vashishtha 912e372c4e markdown: Remove !avatar() and !gravatar() syntax.
This particular commit has been a long time coming. For reference,
!avatar(email) was an undocumented syntax that simply rendered an
inline 50px avatar for a user in a message, essentially allowing
you to create a user pill like:

`!avatar(alice@example.com) Alice: hey!`

---

Reimplementation

If we decide to reimplement this or a similar feature in the future,
we could use something like `<avatar:userid>` syntax which is more
in line with creating links in markdown. Even then, it would not be
a good idea to add this instead of supporting inline images directly.

Since any usecases of such a syntax are in automation, we do not need
to make it userfriendly and something like the following is a better
implementation that doesn't need a custom syntax:

`![avatar for Alice](/avatar/1234?s=50) Alice: hey!`

---

History

We initially added this syntax back in 2012 and it was 'deprecated'
from the get go. Here's what the original commit had to say about
the new syntax:

> We'll use this internally for the commit bot.  We might eventually
> disable it for external users.

We eventually did start using this for our github integrations in 2013
but since then, those integrations have been neglected in favor of
our GitHub webhooks which do not use this syntax.

When we copied `!gravatar` to add the `!avatar` syntax, we also noted
that we want to deprecate the `!gravatar` syntax entirely - in 2013!

Since then, we haven't advertised either of these syntaxes anywhere
in our docs, and the only two places where this syntax remains is
our game bots that could easily do without these, and the git commit
integration that we have deprecated anyway.

We do not have any evidence of someone asking about this syntax on
chat.zulip.org when developing an integration and rightfully so- only
the people who work on Zulip (and specifically, markdown) are likely
to stumble upon it and try it out.

This is also the only peice of code due to which we had to look up
emails -> userid mapping in our backend markdown. By removing this,
we entirely remove the backend markdown's dependency on user emails
to render messages.

---

Relevant commits:

- Oct 2012, Initial commit        c31462c278
- Nov 2013, Update commit bot     968c393826
- Nov 2013, Add avatar syntax     761c0a0266
- Sep 2017, Avoid email use       c3032a7fe8
- Apr 2019, Remove from webhook   674fcfcce1
2020-07-07 10:39:44 -07:00
Aman Agrawal f1cc2ab926 delete_event_notify_user_ids: Fix UnboundLocalError. 2020-07-07 10:33:50 -07:00
Rohitt Vashishtha 4a62cae0b7 node-tests: Log file name and test label on failure. 2020-07-07 10:38:11 -04:00
Rohitt Vashishtha 1d3234b6c6 timerender: Make moment() return static date.
The previous iteration of the test was flaky when we could get
off by one errors between two moment() calls.
2020-07-07 08:04:50 -04:00
sahil839 7079897119 docs: Add steps for configuring message retention policy in the docs.
This commits adds the steps for configuring message retention policy
for an organization and for individual streams in message retention
policy docs.

Fixes #15495.
2020-07-06 18:55:02 -07:00
Alex Vandiver d49e73e087 test-install: Do not install rabbitmq in base install.
The installer does not adjust the node name if the rabbitmq already
exists, and the default node name bakes in the
`zulip-install-bionic-base` hostname.  As such, the resulting LXC
image does not properly start rabbitmq.

Remove rabbitmq, allowing the installer to install and configure it
with a nodename of `zulip@localhost`.  This also lets the installed
image be successfully copied and booted under a new hostname without
breaking rabbitmq.
2020-07-06 18:53:38 -07:00
Anders Kaseorg 5693f63bf8 settings: Convert variable type annotations to Python 3.6 style.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-06 18:44:58 -07:00
Anders Kaseorg ebf7f4d0f6 zthumbor: Rename thumbor.conf to thumbor_settings.py.
So we can apply all our lint checks to it.

Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-06 18:44:58 -07:00
Anders Kaseorg 9900298315 zthumbor: Remove Python 2 residue.
Signed-off-by: Anders Kaseorg <anders@zulip.com>
2020-07-06 18:44:58 -07:00
Sanskar Bajpai bd6fe786cf date_row: Update color via update_table_stream_color.
Previously, the function update_table_stream_color did not update the
color of the date_row when it appeared within a topic.  This was
incorrect because it meant that a small piece of the border to the
left of the message box would be left to the previous color when the
stream color was updated.  This commit resolves the issue by adding an
update.

Fixes #15197.
2020-07-06 18:37:35 -07:00
Alex Vandiver 0d7dbd1b07 puppet: Apply basic PostgreSQL configuration before pg_upgradecluster.
Running `pg-upgradecluster` runs the `CREATE TEXT SEARCH DICTIONARY`
and `CREATE TEXT SEARCH CONFIGURATION` from
`zerver/migrations/0001_initial.py` on the new PostgreSQL cluster;
this requires that the stopwords file and dictionary exist _prior_
to `pg_upgradecluster` being run.

This causes a minor dependency conflict -- we do not wish to duplicate
the functionality from `zulip::postgres_appdb_base` which configures
those files, but installing all of `zulip::postgres_appdb_tuned` will
attempt to restart PostgreSQL -- which has not configured the cluster
for the new version yet.

In order to split out configuration of the prerequisites for the
application database, and the steps required to run it, we need to be
able to apply only part of the puppet configuration.  Use the
newly-added `--config` argument to provide a more limited `zulip.conf`
which only applies `zulip::postgres_appdb_base` to the new version of
Postgres, creating the required tsearch data files.

This also preserves the property that a failure at any point prior to
the `pg_upgradecluster` is easily recoverable, by re-running
`zulip-puppet-apply`.
2020-07-06 18:30:16 -07:00
Alex Vandiver 17002f2a0e puppet: Allow passing an alternate config path to zulip-puppet-apply.
When temporary configuration changes are desired, this lets one set up
an alternate `zulip.conf` to apply while leaving the true one in place.
2020-07-06 18:30:16 -07:00
Alex Vandiver efe2b6e5cd puppet: Switch `zulip-puppet-apply` to argparse.
This allows additional arguments other than `-f` or `--force`.
2020-07-06 18:30:16 -07:00
Priyank Patel 29dd0c485a ci: Move upgrading git for bionic to docker image.
Just reduces the clutter in configuration file for GitHub Actions.
2020-07-06 18:09:15 -07:00
Priyank Patel c908278237 ci: Install moreutils in the docker image. 2020-07-06 18:09:15 -07:00
Priyank Patel 650ec29859 github-actions: Use stratergy and if to deduplicate steps.
This is a fine solution short-term until github implements the
yaml anchors support. The limitation of this method is that we
cannot re-use most of the steps again for production install test
builds.

Thanks, Anders for this solution.

Verifying everything is migrated correctly is a pain. This script
ensures everything is done correctly (previous commit message
contains explainations for the steps being ignored if; in case
of github-actions steps they are ignored because they are actions
specific):
"""
This script prints out the ignore steps first. Then
prints out each step of both circle and actions side-by-side.
One step is out of order for bionic but verfying correction is
still easier. Format:
Actions: Install dependencies
Circle CI: install dependencies
....
"""
import yaml

with open('.circleci/config.yml') as f:
    circleci_config = yaml.safe_load(f)

with open('.github/workflows/zulip-ci.yml') as f:
    actions_config = yaml.safe_load(f)

circle_bionic_steps = []
circle_focal_steps = []
actions_bionic_steps = []
actions_focal_steps = []

"""
We ignore casper artifact upload, save_cache, and
store_tests_reports steps.
"""
def get_circleci_steps(job, arr):
    for step in circleci_config['jobs'][job]['steps']:
        if isinstance(step, str):
            arr.append(step)
            continue

        step_name = step.get('run', {}).get('name', False)
        if not step_name:
            if step.get('restore_cache'):
                key = step['restore_cache']['keys'][0].split('.')[0]
                step_name = f'<restore-cache> {key}'
            elif step.get('store_artifacts', False):
                destination = step['store_artifacts']['destination']
                step_name = f'<store-artificats> {destination}'

                if destination == 'casper':
                    \# This is no longer needed
                    print('Ignoring step:')
                    print(step)
                    print()
                    continue
            else:
		"""
                We don't care about save_cache; github-actions
                does this automatically, and store_tests_reports
                is circelci timing specific.
		"""
                print('Ignoring step:')
                print(step)
                print()
                continue

        if step_name != 'On fail':
            arr.append(step_name)

get_circleci_steps('bionic-backend-frontend', circle_bionic_steps)
get_circleci_steps('focal-backend', circle_focal_steps)

""" We ignore there steps specific to github-actions"""
for step in actions_config['jobs']['focal_bionic']['steps']:
    BOTH_OS = 'BOTH_OS'
    if_check = step.get('if', BOTH_OS)
    step_name = step.get('name')

    if step_name is None:
        step_name = step['uses']

    if (
        step_name == 'Upgrade git for bionic' or
        step_name == 'Add required permissions' or
        step_name == 'Move test reports to var'
    ):
        print('Ignoring step:')
        print(step)
        print()
        """These are github-actions specific; see comments"""
        continue

    if if_check == BOTH_OS:
        actions_bionic_steps.append(step_name)
        actions_focal_steps.append(step_name)
    elif 'is_bionic' in if_check:
        actions_bionic_steps.append(step_name)
    else:
        actions_focal_steps.append(step_name)

bionic = zip(circle_bionic_steps, actions_bionic_steps)
focal = zip(circle_focal_steps, actions_focal_steps)

print('Bionic steps:')
for (circle_step, actions_step) in bionic:
    print(f'CircleCI: {circle_step}')
    print(f'Actions: {actions_step}')
    print()

print('Focal steps:')
for (circle_step, actions_step) in focal:
    print(f'CircleCI: {circle_step}')
    print(f'Actions: {actions_step}')
    print()
2020-07-06 18:09:15 -07:00
Priyank Patel ad5eb68ee1 github-actions: Add focal backend job.
No new sets of change in this commit that are explained two commits
before this one, which add bionic-backend-frontend.
2020-07-06 18:09:15 -07:00
Priyank Patel e834439d82 provision: Do service <service-name> start instead of restart.
Doing service memcached start instead of restart fixed an issue on
focal build in GitHub actions, where it exits with code 1 when it
is done twice.It is done first in Install Dependencies step and then
again in last step where we call tools/ci/setup-backend again which
runs provision.

Furthermore, I don't belive there is a technical reason we use
restart over start; rather I think it was just a random choice with
the intend to just start the services in CI. I traced the code back
to commit 1f2f497cab if it helps.
Looking at the source code of memcached, the step that's failing is:
  start-stop-daemon --stop --quiet --oknodo --pidfile $PIDFILE
which is equivilent to: service memcached stop, we can rule out the
service memcache start since it works. Ideally, we do figure out and
solve the issue of why memcached fails when executing service
memcached stop but I am not equipped with debugging it. And this
workaround seems reasonable rather than a "hacky" solution.

For the relevant code in memcached see:
https://github.com/memcached/memcached/blob/master/scripts/memcached-init.

Finally, the change to the rest of services is for consistency.
2020-07-06 18:09:15 -07:00
Priyank Patel 7ff1fd5923 github-actions: Add bionic frontend and backend test job.
Some noteable diffrence from circleci:
   - We upgrade git to newer version (reason explained in comments)
   - We set HOME to /home/github (also explained in comments)
   - Adjust permissions (... comments)
   - Minor changes to step names and cache keys.
   - We don't need to port the save_cache steps they are done
     automatically in actions. And, we did not port the
     store_test_results step which is circleci specific.
   - We didn't port the notify_failure step yet (see the TODO).
2020-07-06 18:09:15 -07:00
arpit551 a2428017e9 audit_log: Log RealmAuditLog in do_set_realm_authentication_methods.
Log RealmAuditLog in do_set_realm_authentication_methods and added
tests for it.
2020-07-06 17:38:20 -07:00
arpit551 ba92666dbd audit_log: Log RealmAuditLog in do_set_realm_property.
Log RealmAuditLog in do_set_realm_property and do_remove_realm_domain.
Tests for the changes are written in test_events because it will save
duplicate code for test_change_realm_property.
2020-07-06 17:38:19 -07:00