Commit Graph

24092 Commits

Author SHA1 Message Date
Greg Price 23e6a2e579 puppet: Update memcached config to turn on this decade's technology.
We've been running this change on zulipchat.com for a couple of months
now.  Before then, we used to regularly get exceptions like this:

     File "./zerver/views/messages.py", line 749, in get_messages_backend
       setter=stringify_message_dict)
     File "./zerver/lib/cache.py", line 275, in generic_bulk_cached_fetch
       cache_set_many(items_for_remote_cache)
     File "./zerver/lib/cache.py", line 215, in cache_set_many
       get_cache_backend(cache_name).set_many(items, timeout=timeout)
     File "/home/zulip/deployments/2017-09-28-21-04-12/zulip-py3-venv/lib/python3.5/site-packages/django/core/cache/backends/memcached.py", line 150, in set_many
       self._cache.set_multi(safe_data, self.get_backend_timeout(timeout))
   pylibmc.Error: error 48 from memcached_set_multi

This error means memcached was unable to find space for the new value.
You might think that because memcached provides an LRU cache, this
shouldn't happen because it would just evict something... but in fact
  * memcached splits its data into "slabs" by object size, and
  * until recently, once a 1MiB "chunk" is allocated to a given "slab"
    i.e. size class, it wouldn't be reclaimed to allocate to another.

So once the cache has been filled up with objects of some distribution
of sizes, if some objects come in that would go in a different size
class, we have no chunks for that size class / slab, and can't get one.
And that's exactly what was happening on zulipchat.com.

Useful background can be found in
  https://github.com/memcached/memcached/wiki/ServerMaint#slab-imbalance
  https://github.com/memcached/memcached/wiki/ReleaseNotes1411
  https://github.com/memcached/memcached/wiki/ReleaseNotes1425
  https://github.com/memcached/memcached/wiki/ReleaseNotes150
We're already running v1.4.25, which provides an "automover" that should
be well equipped to fix this; v1.5.0 turns it on by default.

With this commit, adopt the "modern start line" recommended in the
release notes for our v1.4.25, including turning on the automover.
2018-02-08 16:34:49 -08:00
Eeshan Garg 4414f2b24b webhooks/beeminder: Use create-a-bot.md macro. 2018-02-08 16:22:55 -08:00
Eeshan Garg ab700fb662 markdown: Add a macro with instructions for creating a bot. 2018-02-08 16:22:55 -08:00
Rhea Parekh 48640fd28f slack importer: Suppress logger output from the unit tests. 2018-02-08 16:21:35 -08:00
Vishnu Ks 680983d10e vagrantfile: Workaround for the atlas.hashicorp.com redirect removal.
This workaround is only for new users. Users who are/were using old
versions of Vagrant has to manually change the value of box
metadata_url. See https://github.com/hashicorp/vagrant/issues/9442
2018-02-08 13:54:19 -08:00
Tim Abbott 9249454518 update-authors-json: Fix handling deleted GitHub accounts.
Apparently, we've now had the first time one of our contributors had
their account deleted (at least, the author page for the contributor
who has 21 commits in python-zulip-api now 404s).
2018-02-08 12:17:20 -08:00
Greg Price 9e70026741 settings: Slightly clarify LDAP setup docs.
There's kind of a lot of prose in this LDAP section; and empirically
this bit about zulip-secrets.conf is easy to miss.
2018-02-08 10:05:25 -08:00
Robert Hönig 537203102a api tests: Update remove-subscriptions fixture.
The remove-subscriptions API client response
changed with python-zulip-api:67cdb6f
2018-02-08 09:37:41 -08:00
Brock Whittaker e82564c82c dark-mode: Fix bug with open graph previews.
This fixes a bug where the open graph preview bottom fade is dark
rather than white when not in dark mode, which results in a heavy
dark faded line at the bottom of the description.
2018-02-08 09:22:58 -08:00
Eeshan Garg 61e4fc3b74 api/remove-subscriptions: Make fixtures/examples testable.
api/remove-subscriptions now uses the Markdown extension in
bugdown/api_code_examples to generate code example and fixtures
from tests.
2018-02-07 15:58:57 -08:00
Eeshan Garg f4cb5e70ed api/add-subscriptions: Make fixtures/examples testable.
api/add-subscriptions now uses the Markdown extension in
bugdown/api_code_examples to generate code example and fixtures
from tests.
2018-02-07 15:58:57 -08:00
Eeshan Garg 09721e9ec3 bugdown/api_code_examples: Support multiple fixtures per doc.
This commit adds support for passing in an argument to the macro
"call" to explicitly specify a fixture to render, like so:

{generate_code_example|doc_name|fixture(stream_message_with_args)}
2018-02-07 15:58:57 -08:00
Eeshan Garg 7ae704d1bb api/get-all-streams: Render multiple examples from single test method.
api/get-all-streams - make use of the api_code_example Markdown extension's
feature of recursively extracting multiple code examples from a single
test method.
2018-02-07 15:58:57 -08:00
Eeshan Garg 0cca39d1fc bugdown/api_code_examples: Support multiple examples in one test.
This commit adds support for recursively extracting multiple code
examples from a single test method.
2018-02-07 15:58:57 -08:00
Eeshan Garg ffe930d50d bugdown/api_generate_examples: Replace 'method' with 'example'.
To generate a code exammple,

{generate_code_example|<api_doc_md>|example} sounds better and
more intuitive than,

{generate_code_example|<api_doc_md>|method}
2018-02-07 15:58:57 -08:00
Greg Price ccb8c79edb test_docs: Fix a now-brittle test.
The count of integrations is automatically computed now, so this will
change every time we add 10 more.  Just stop asserting on the number.

Thanks to @hackerkid for spotting the issue.
2018-02-07 13:01:42 -08:00
Josh Mandel 15eaafa50f Add underscores to thumbs_up and thumbs_down emoji 2018-02-07 11:55:02 -08:00
Ricky db152da7c9 webhooks/beeminder/doc.md: Add instructions for creating a bot. 2018-02-07 10:03:31 -08:00
Ricky 77760d9f6c webhooks/beeminder: Update sample message screenshot. 2018-02-07 10:03:31 -08:00
Ricky 1303248b0b integration: Add statuspage.io webhook integration. 2018-02-07 09:59:12 -08:00
Eeshan Garg d7903d25e9 pypi_packages: Upgrade to release 0.4.1. 2018-02-07 09:52:51 -08:00
Greg Price c80802ff1e bots: Clean up create_realm_internal_bots.
This code duplicated the code in setup_realm_internal_bots, with some
added logic to avoid trying to create the same bot twice.  That logic
was buggy so that it would never work at all -- it subtracted a set of
UserProfile objects from a set of email strings -- so it looked like
the command might blow up when run after the users already existed.

In fact, the buggy logic wasn't necessary, because the work the
command does after it is idempotent -- in particular `create_users`,
within its subroutine `bulk_create_users`, already filters out users
that already exist.  So just cut the buggy stuff out, deduplicate the
rest with `setup_realm_internal_bots`, and document that invariant on
the latter.

While we're here, in the common case bail early without doing any
per-realm work in Python, since we're running this on every upgrade.
2018-02-06 18:59:57 -08:00
Greg Price a9752c5163 left-sidebar: Add back link underlines on hover, just for topics and PMs.
These were removed in 953ee778f, along with the stream names and
right-sidebar user names mentioned in the commit message.  But unlike
those, these don't get any background change on hover, so we need some
way to show the hover.

We can probably still do something that looks nicer than this, but
it's better than not having it.
2018-02-06 17:17:14 -08:00
Greg Price 52c57056cb left-sidebar: Strengthen global filter icons partway back.
This makes them roughly the same visual weight as the (non-muted)
stream icons below -- though that's imprecise because the latter
varies with color.  Tested in both normal and dark-mode.
2018-02-06 17:02:12 -08:00
Greg Price 3c1349b199 stream settings: Clean up a permissions message. 2018-02-06 16:37:18 -08:00
Aditya Bansal 643074edb2 notifications: Add test for realm_name_in_notifications setting. 2018-02-07 05:09:46 +05:30
Rhea Parekh 83a7fd84ab slack importer: Import primary owner user first.
According to https://get.slack.help/hc/en-us/articles/201912948-Owners-and-Administrators,
only one Primary owner of a slack organsation exists. This allocates the first id
to the Primary owner and hence makes sure that the primary owner is imported first.
Added tests for the same.
2018-02-06 14:48:30 -08:00
Rhea Parekh 052e3e1540 slack importer: Change organization admin mappings.
Map 'Primary owner', 'owner' and 'admin' to 'organization admin'.
Added tests for the same.
2018-02-06 14:48:30 -08:00
Rhea Parekh b3b6023230 slack importer: Always map 'is_staff' to false in user data.
"staff" is only for server administrators, which doesn't exist in Slack.
Hence, this should always be false.
2018-02-06 14:48:29 -08:00
Greg Price 756bbef1f2 Revert "settings: Check if xhr response text include ':' or not."
This reverts commit dd406e87e.  Whatever awkward error message might
have motivated that change, we can fix in a more targeted way when we
see it again.  This version would delete the potentially-informative
context the caller provided in `response` -- even just "Failed" --
based on whether an error message happened to contain a colon
somewhere in it, which is sure to end up feeling unpredictable.
2018-02-06 13:44:10 -08:00
Umair Khan 764cd977d6 test-queue-worker-reload: Increase delay to 1.3s.
See 625939 for more information. In short, the purpose of this delay is
to give autoreload code enough time to touch every watched file at least
once before the change is made.
2018-02-06 10:35:20 -08:00
synicalsyntax 9bb267a016 settings: Force strings to wrap in Filter settings table.
Fixes #8140.
2018-02-06 09:00:11 -08:00
Rishi Gupta 77817410a1 settings: Change wordings of a few notification settings.
The last bit of #8059 (merged as b875fe07) didn't get merged, so just adding
it as a followup.

The "Other notification settings" section will also eventually house the
"pick your notification sound" setting, which is why it isn't called "Other
email settings" or similar.
2018-02-06 08:58:52 -08:00
Robert Hönig 146b833454 marketing: Update integration count.
This also automizes the integrations count update for
the /integrations page.
2018-02-06 08:50:41 -08:00
Greg Price 52f1c57c2a migrations: Fix mypy error in placeholder migration. 2018-02-05 18:03:07 -08:00
Roman Godov b875fe07eb settings: Added setting to turn on and off realm name in email subject.
Users having only account in one realm will not be distracted by realm
name in subject lines of every email.  Users who have multiple
accounts in realms can turn this setting on and receive a
corresponding realm name in email's subject.

Tweaked by tabbott to rebase and address a few small issues.

Fixes #5489.
2018-02-05 18:01:54 -08:00
Aditya Bansal ecbb763e79 notifications: Handle an edge case in relative_to_full_url.
For messages where the entire rendered body is a message_inline_image
object, we actually don't display any text and just display the
image. These messages may have links to images which might or might
not be internal to Zulip but in both cases there is a chance of this
links being broken when accessed by an email server like Gmail that
doesn't possess the recipient user's cookies.

We don't want to have ugly looking broken images displayed in email
notifications.  So we patch this by inserting a replacement for the
`message_inline_image` block in which we essentially replace the
content with the textual link.

Edited for clarity by tabbott.
2018-02-05 17:30:55 -08:00
Greg Price 6f128279e8 migrations: Add reverser for emoji_alt_code migration.
This is easy to do, and prevents this feature from getting a server
admin stuck in potentially a pretty uncomfortable way -- unable to
roll back a deploy.
2018-02-05 16:55:21 -08:00
Greg Price ca1129dea3 Revert "Streams: Disallow certain markdown characters in stream names."
This reverts commit dd1b14378b.

Deleted a conflicted test entirely.
2018-02-05 16:55:21 -08:00
Greg Price 44a0cd77cb Revert "UserProfile: Disallow markdown character in the user profile name."
This reverts commit d30fb5a341.
2018-02-05 16:55:21 -08:00
Greg Price 0b3a414be1 Revert "migrations: Replace special chars in stream & user names with space."
This reverts commit acebd3a5e, as well as a subsequent fixup commit
0975bebac "quick fix: Fix migrations to be linear."

These changes need more work and thought before they're ready to
deploy on any large established Zulip server, such as zulipchat.com.
See discussion on #6534.

In place of the removed migration, leave behind a placeholder so
`manage.py migrate` doesn't get confused on installs where it was
already applied.
2018-02-05 16:55:21 -08:00
Shubham Dhama 9a75a10d48 tests: Test invalid "default stream group" name. 2018-02-05 16:33:42 -08:00
Shubham Dhama dffedeb053 tests: Test bots deactivation with owner deactivation.
This also cover the `zerver.lib.actions.do_deactivate_user` when
`_cascade=true`.
2018-02-05 16:33:42 -08:00
Brock Whittaker 06f3cb2b78 message-feed: Add some styling to open graph previews.
This adds some styling to make the open graph previews look a bit nicer,
including:

1. Adding a bottom fading gradient to slowly fade out text that is out
of bounds rather than chopping it off.
2. Using font anti-aliasing to make the characters appear smoother.
3. Increasing the font size of the title to give it prominence.
4. Changing the height to 80px from 70px.
2018-02-05 16:27:31 -08:00
Umair Khan c415cc74d7 validate_stream_message_address_info: Add i18n tags.
Fixes #7076
2018-02-05 16:26:24 -08:00
Brock Whittaker 1a93cd4ffe panels: Give admins a red warning banner when email isn't configured.
Also refactor the "panels"/banner code to be a bit clearer about how
it's supposed to generically work, using [data-process] as a uniquely
identifying marker.

Fixes: #8166.

[greg: rebased and squashed a series of fixup commits.]
2018-02-05 16:00:52 -08:00
Brock Whittaker fe3a5665da panels: Rename "desktop_notifications_panel" => "panels".
The original code made a 3/4-hearted effort to generically accommodate
more banners/"panels" later, but named itself after the first one made.

[greg: expanded commit message.]
2018-02-05 16:00:52 -08:00
Brock Whittaker f485abf522 page_params: Pass through settings.WARN_NO_EMAIL, as `warn_no_email`.
[greg: fixed test]
2018-02-05 16:00:33 -08:00
Tim Abbott 64cb05be67 import: Update the documentation slightly. 2018-02-05 14:53:07 -08:00
Rhea Parekh 78e0a2613f slack importer: Add documentation for Zulip default stream. 2018-02-05 14:46:39 -08:00